Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 20208 - Mirrorselect enhancements
Summary: Mirrorselect enhancements
Status: RESOLVED FIXED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Current packages (show other bugs)
Hardware: All Linux
: High enhancement (vote)
Assignee: John Mylchreest (RETIRED)
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2003-04-30 06:35 UTC by Narada Sage
Modified: 2003-10-01 12:11 UTC (History)
1 user (show)

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Narada Sage 2003-04-30 06:35:40 UTC
Here is a request for an enhancement to app-admin/mirrorselect.  At the 
moment, it only supports GENTOO_MIRRORS="".  It should support additionally 
SYNC="".  The list of download and sync mirrors can be placed in a text file 
that is updated with 'emerge sync'.  Perhaps this will ease the load on sync 
servers by encouraging the use of local sync mirrors.  Many local sync mirrors 
remain however out of date so only reliable ones should be placed on the list.
Comment 1 Narada Sage 2003-05-10 07:40:50 UTC
Having tried mirrorselect 0.3 here are a few thoughts.  Many thanks.
- In mirrorselect -i the mirrors for all countries are scattered in random order.  Perhaps, mirrors for each country can be grouped together to make it easier to choose from.
- I notice that the program makes use of a backup file named make.conf.old.  Although this is all well, I should point out that I use that file name for my own backup.  It probably doesn't warrant change but I thought I should mention it.
- During mirrorselect -a -sX -o there is a certain delay without any comments being printed to the screen.  It gave the impression for a moment that it had crashed.  Adding comments similar to mirrorselect -a -sX would be nice.
Comment 2 John Mylchreest (RETIRED) gentoo-dev 2003-05-11 15:48:57 UTC
Hi, thanks for trying it out.

In response to your queries.
I will look into adding sort on country for -i now.
The backup file, imo is suitable as it as, as no matter what I call it, the same situation may arise.
and the -o, I feel I should document this better - so I shall do so now.
-o is used as a non-versbose way to redirect the lines, rather than a normal use flag.
for example, as part of the installation, before you have chrooted into the system and still currently on the livecd you need to redirect to a non-standard file. such as:

mirrorselect -a -s4 -o >> /mnt/gentoo/etc/make.conf

this is why it is silent. hope that clears a few things up. any ideas welcome :D
Comment 3 John Mylchreest (RETIRED) gentoo-dev 2003-05-11 18:29:43 UTC
preliminary rsync support added to mirrorselect-0.4
this is in portage marked unstable.

please test - thanks :)
Comment 4 Narada Sage 2003-05-12 10:33:23 UTC
Hello.  Excellent work on mirrorselect and thank you for participating in feedback on enhancements.  Here is my evaluation of mirrorselect 0.4.

(1) At present, /usr/sbin/mirrorselect can be run as normal user with all possible parameters.  The Save button can also be selected.  Luckily, having tried every parameter as normal user I was unable to make any changes to make.conf so it is secure although it did say that changes were made.  However, I did receive some interesting proposals from the utility regarding overriding permissions.  Here are some examples.

[normaluser]$  /usr/sbin/mirrorselect -a -s1
Downloading a list of mirrors...
Running netselect on the serverlist, this may take a moment...
Found: http://194.83.57.2/sites/www.ibiblio.org/gentoo/
Updating /etc/make.conf...
mv: overwrite `/etc/make.conf.old', overriding mode 0644? y
mv: cannot move `/etc/make.conf' to `/etc/make.conf.old': Permission denied
Mirrors set successfully
rm: remove write-protected regular file `/etc/make.conf.old'? y
rm: cannot remove `/etc/make.conf.old': Permission denied

This can be corrected in two ways.  One is to add a root uid check and refuse execution on the command prompt regardless of parameters used.  Another is to do what UFED devs have done which is that when executed as a normal user one can only view use flags and the 'Save' button changes to 'Read Only/No Saving'.  Personally, mirror information should be of no significance to the non-root user so I would prefer that it is barred from normal user use completely.

(2)When executing mirrorselect -a -s6 -o the list of mirrors are shown in double quotes.  However, there is a space before the closing double quote.  This can be removed presumably.

[root]$ mirrorselect -a -s1 -o
GENTOO_MIRRORS="http://ftp.belnet.be/mirror/rsync.gentoo.org/gentoo/ "

(3)The parameters currently in use work perfectly.  However, I for one find them a little unintuitive and confusing.  Here is a new parameter structure I have in mind that I find to be more logical.

mirrorselect
-i	interactive 
	-g	gentoo_mirrors
	-s	sync_mirrors
-a	automatic
	-g	gentoo_mirrors
	-s	sync_mirrors
-o output

I hope the example usage is relatively clear here; concatenated usage may also be possible such as -ig -as1 but would also work separately.  Of course such things are usually quite subjective and I will leave this at your discretion.

(4) On gentoo-dev mailing list recently a message was posted by Sven Vermeulen saying that every instance of 'emerge rsync' in docs would now be replaced with 'emerge sync'.  In mirrorselect, I notice that rsync is utilised in parameters and in interactive mode.  However in make.conf SYNC="" is used.  I suggest that by convention SYNC should be used everywhere for consistency.

(5) I'd like to recommend an excellent UK sync mirror to be added to the list: rsync://rsync.uk.gentoo.org/gentoo-portage.  The speeds are blisteringly fast and the mirror always remains in date and as such is recommended.  I realise this may not be in the mirrors.xml file downloaded so once again I'll leave this to the discretion of the devs to add it.  On a side note, I'm not convinced that the mirrors list for distfiles or sync are complete.  I can go through the national mirror lists and prepare a complete world list if need be.

(6) When -s is used without a number it says -s is an invalid option.  Perhaps this should say -s must be used in conjunction with a number.

[root]$ mirrorselect -a -s -o
-s is an invalid option

(7) Eventually, a man page entry would be gratefully received for mirrorselect which is currently missing. A little 'Help' tab on the bottom right alongside 'Cancel' would be nice just like ufed.

I hope these comments have been beneficial.  I have been verbose so that nothing gets left out.  Hope to hear your thoughts.  Many thanks.
Comment 5 Narada Sage 2003-05-12 10:48:58 UTC
As an amendment to suggestion (6) running mirrorselect -a -s with no number should perhaps have the same result as mirrorselect -a with no 's' option which is to return the top one mirror.
Comment 6 John Mylchreest (RETIRED) gentoo-dev 2003-05-12 19:02:24 UTC
mirrorselect-0.5 is hitting portage unstable.
this boasts a massive code-restructure, and hopefully addresses all of your issues.
can you please confirm.

thanks
Comment 7 Narada Sage 2003-05-12 21:44:58 UTC
Mirrorselect 0.5 is looking very promising indeed!  Here are some further thoughts.

(1) Once again I have proposed slightly different usage as I find the current parameters still a little asymmetric.  I had made a mistake in my previous suggestion but this should be right.  I'll leave this at your discretion.  I'm going by the thought that -i shouldn't give preference to distfile or sync.

-d: opens UI for manual selection of distfile mirrors
-s: opens UI for manual selection of sync mirrors
-d -a -nX: automatic console retrieval and selection of X download mirrors
-r (-a -sX): not supported
-d -a -o: automatic console redirection to stout or specified file

(2) The message: No mirrors have been found. Exiting. should be changed to: No mirrors have been selected. Exiting. since in all contexts that is what the utility means to say.  This message appears when mirrorselect -r -i is started and 'Esc' is pressed or 'Save' / 'Cancel' is depressed without selecting any mirrors or mirrorselect -a -s0 is used.

(3) The utility UI should be enlarged to make better use of the space like ufed so that less scrolling is required.  When browsing distfile mirrors one has to scroll quite a bit.

(4) For consistency just as ufed enters the variable into make.conf in the USE section of make.conf distfile and sync mirror variables can be entered just after the GENTOO_MIRRORS="" section in that order.

(5) A grammatical change: when executed as normal user it should say 'This application must be run (ran) as root.'  Also, I propose that the usage/help information either be modified in light of above usage or be rewritten to be slightly more verbose and clearer or merely amending punctuation and capitalisation in current text.  I am prepared to help in this respect if need be.

(6) There should be upper and lower limits for mirrorselect -a -sX and these should be specified in help text.  At the moment any number from 0 to a high number is possible.  With 0 it proceeds to download mirrorlist and run netselect but doesn't select any.  Perhaps zero should be disabled.  Also, given that there's a maximum of 49 mirrors the upper limit can either be the maximum number of mirrors (49) or it can be a much lower number like 10 to save typing in ridiculously high numbers.  Either way, specifying higher than upper limit should return user to help.

(Here are a few very wild thoughts that are probably too crazy to act on but are there out of interest.  Can sync and distfiles mirrors be integrated into the same GUI and when sync and distfile mirrors are selected and save pressed the two variables are entered into make.conf?  Would this require a migration to ncurses from dialog?)

Many thanks and once again excellent work.
Comment 8 Narada Sage 2003-05-12 21:51:20 UTC
A mistake is in my parameters again(!)  -r (-a -sX): not supported should have been -r (-a -nX): not supported.  For completeness even if it isn't adopted.  Thanks.
Comment 9 John Mylchreest (RETIRED) gentoo-dev 2003-05-13 04:28:43 UTC
Hey there.
Thanks for testing, im glad things are ok for you.

In regard to your requests, I shall quote my opinions on your points in turn:

1) Currently download mirrors are favoured as rsync policy hasnt been clarified as to how much, or how little mirrorselect can handle. RSYNC is still currently only in preliminary support, and there is no automatic selection of rsync mirrors. If this changes, and mirrorselect becomes very supporting for SYNC="" I shall refer to this bug for ideas on changing parameters.

2) changed in my local copy, and should be fixed in the new release.

3) This is currently on my to-do list. although I dont want to do too much on the unstable tree, without stable being updated to a more recent version. I propose this one to go to stable after sufficient testing.

4) I believe somebody offered to submit a patch for this to me. If this is the case I shall incorporate it for the next release, if not, It is on my to-do list.

5) I have updated the error message. If you would like to describe what you mean in regard to the usage, I will look to update it.

6) I can certainly put in a lower limit of 1, so if a value under 1 is entered, it will use 1. As far as an upper limit is concerned, I cannot determine the number of mirrors I can limit to until after a download, so I feel that I should leave this open, and once I know the number of mirrors, and the value they entered is higher, cut their requested number down to the number of mirrors.

In all honesty they should never use the program to return such a huge number of mirrors.

In regard to the GUI, I am determined to keep the GUI as simple, and leightweight as possible, as this isnt a commonly ran application, and typically people would use an automated option. If this crops up again though, I will certainly look into it.

I hope for the next version to be released within a month, after this has had suitable testing to move to stable. Hopefully you will see the changes of this in 0.6, thanks for helping out :)

If you can think of anything else please feel free to submit it.
in the mean time I shall change this bug to TEST_REQUEST.

Happy Mirrorselecting :)
Comment 10 John Mylchreest (RETIRED) gentoo-dev 2003-10-01 12:11:28 UTC
lost resolutions, re-closing