Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 62899 - 'mirrorselect' generate invalid output (adds binary data to the output file)
Summary: 'mirrorselect' generate invalid output (adds binary data to the output file)
Status: RESOLVED FIXED
Alias: None
Product: Portage Development
Classification: Unclassified
Component: Tools (show other bugs)
Hardware: All Linux
: High normal (vote)
Assignee: Portage Tools Team
URL:
Whiteboard:
Keywords:
: 66115 (view as bug list)
Depends on:
Blocks:
 
Reported: 2004-09-05 03:06 UTC by Patrik Knakal
Modified: 2004-10-12 03:33 UTC (History)
1 user (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
generated result (mirrorselect-result.gz,409 bytes, application/octet-stream)
2004-09-05 03:09 UTC, Patrik Knakal
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Patrik Knakal 2004-09-05 03:06:18 UTC
Using Gentoo2004.2 minimal installation LiveCD. In the documentation there is section about using the 'mirrorselect' script. This script generate incorrect code - there should be only GENTOO_MIRRIRS variable and its value, but there is binary data before this variable.

Next is probably problem, that first selected mirror's protocol has three slashes instead of two.

So, this can be fixed by removing the binary data from the generated strings and remove third slash from the first server definition.

Reproducible: Always
Steps to Reproduce:
1.start system from the mini LiveCD
2.after all necessary setups start 'mirrorselect' script
3.view make.conf file

Actual Results:  
g
Comment 1 Patrik Knakal 2004-09-05 03:06:18 UTC
Using Gentoo2004.2 minimal installation LiveCD. In the documentation there is section about using the 'mirrorselect' script. This script generate incorrect code - there should be only GENTOO_MIRRIRS variable and its value, but there is binary data before this variable.

Next is probably problem, that first selected mirror's protocol has three slashes instead of two.

So, this can be fixed by removing the binary data from the generated strings and remove third slash from the first server definition.

Reproducible: Always
Steps to Reproduce:
1.start system from the mini LiveCD
2.after all necessary setups start 'mirrorselect' script
3.view make.conf file

Actual Results:  
gÑ/%¦
sõÔïYÂc$ÙYpm[V§õ2OãäåüÞ`/÷cÿ¥QMÒ¸xªÕr3v[õ
Kt1p¹CÉMÒÕxðCßè²ÔZ÷ò-Yé_j#N/<ó@²$«®^P¾iqö
È^VrCÀñÈÂYGo0ªî+òP:(ÎG::úVsù]i±PF}Äí´3)í¦
66×GENTOO_MIRRORS="ftp:///ftp-stud.fht-esslingen.de/pub/Mirro
rs/gentoo/ ftp://ftp.ipv6.uni-muenster.de/pub/linux/distribution
s/gentgÑ/%¦
sõÔïYÂc$ÙYpm[V§õ2OãäåüÞ`/÷cÿ¥QMÒ¸xªÕr3v[õ
Kt1p¹CÉMÒÕxðCßè²ÔZ÷ò-Yé_j#N/<ó@²$«®^P¾iqö
È^VrCÀñÈÂYGo0ªî+òP:(ÎG::úVsù]i±PF}Äí´3)í¦
66×GENTOO_MIRRORS="ftp:///ftp-stud.fht-esslingen.de/pub/Mirro
rs/gentoo/ ftp://ftp.ipv6.uni-muenster.de/pub/linux/distribution
s/gentoo ftp://ftp6.uni-erlangen.de/pub/mirrors/gentoo http://ww
w.die.unipd.it/pub/Linux/distributions/gentoo-sources/ http://ft
p6.uni-erlangen.de/pub/mirrors/gentoo"oo
ftp://ftp6.uni-erlangen.de/pub/mirrors/gentoo http://ww
w.die.unipd.it/pub/Linux/distributions/gentoo-sources/ http://ft
p6.uni-erlangen.de/pub/mirrors/gentoo"

Expected Results:  
GENTOO_MIRRORS="ftp://ftp-stud.fht-esslingen.de/pub/Mirro
rs/gentoo/ ftp://ftp.ipv6.uni-muenster.de/pub/linux/distribution
s/gentoo ftp://ftp6.uni-erlangen.de/pub/mirrors/gentoo http://ww
w.die.unipd.it/pub/Linux/distributions/gentoo-sources/ http://ft
p6.uni-erlangen.de/pub/mirrors/gentoo"oo
ftp://ftp6.uni-erlangen.de/pub/mirrors/gentoo http://ww
w.die.unipd.it/pub/Linux/distributions/gentoo-sources/ http://ft
p6.uni
Comment 2 Patrik Knakal 2004-09-05 03:09:40 UTC
Created attachment 38956 [details]
generated result
Comment 3 David Owen 2004-09-15 09:01:32 UTC
Running the same ISO and same procedure, got the same problem.  Line 187 of /usr/sbin/mirrorselect is currently:

wget -t 1 -C off -T 25 -O - ${i}/distfiles/mirrorselect-test 2>/dev/null

Perhaps it should be:

wget -t 1 -C off -T 25 -O - ${i}/distfiles/mirrorselect-test 2>&1 >/dev/null
Comment 4 Michaelian Ennis 2004-09-22 10:28:07 UTC
Confirming on universal 2004.2 iso
same line 187
wget -t 1 -C off -T 25 -O - ${i}/distfiles/mirrorselect-test 2>/dev/null

Causes env-update to complain and emerge will not run.  Of course removing the garbage from make.conf is an easy fix if your not new at this.
Comment 5 Martin Guy 2004-09-27 08:41:56 UTC
Confirmed installing from the stage3 i686 tarball 2004.2 today from the net,
I get 4.5MB of binary data in make.conf starting when it tests the first mirror, gentoo.osuosl.org

Are linux-haters setting up bogus mirrors to break your distro?

One "solution" and extra defence would be to change the mirrorselect line in
the handbook to grep for the lines you DO want rather than grepping out a
single line that you don't.

Since every valid line starts with GENTOO_MIRROR= and ends with ",

   mirrorselect bla bla | grep '^GENTOO_MIRRORS=.*"$' >> ... should do it,

If it gets garbage (aGAIN! :) that will give an empty output, which
should still work by falling back to the default.
 
Comment 6 Hiel Van Campen 2004-09-27 13:05:35 UTC
1. bug 65498 is a duplicate.
2 the binary data being added is the GOD DAMN ncurses graphic being pasted in to make.conf which borks it. If you run mirrorselect in an xterminal you can better see it. WHy or how it happens i dont know. But after it happens if you try to "less make.conf" I got the bottom half of the grapic and locked up "less". It  was 10 months a go when I first reported it or added to a bug report. I didnt think to mention running it in X and useing less. Hope this helps 
 Portage 2.0.51_rc3 (default-linux/x86/2004.2/gcc34/2.6, gcc-3.4.2, glibc-2.3.4.20040808-r0, 2.6.8-gentoo-r2 i686)
=================================================================
System uname: 2.6.8-gentoo-r2 i686 AMD Athlon(tm) XP 2000+
Gentoo Base System version 1.5.2
Autoconf: sys-devel/autoconf-2.59-r4
Automake: sys-devel/automake-1.8.5-r1
Binutils: sys-devel/binutils-2.15.90.0.1.1-r3
Headers:  sys-kernel/linux26-headers-2.6.8.1
Libtools: sys-devel/libtool-1.5.2-r5
ACCEPT_KEYWORDS="x86 ~x86"
AUTOCLEAN="yes"
CFLAGS="-march=athlon-xp -fomit-frame-pointer -O2 -falign-jumps -pipe"
CHOST="i686-pc-linux-gnu"
COMPILER=""
CONFIG_PROTECT="/etc /usr/X11R6/lib/X11/xkb /usr/kde/2/share/config /usr/kde/3.3/env /usr/kde/3.3/share/config /usr/kde/3.3/shutdown /usr/kde/3/share/config /usr/lib/mozilla/defaults/pref /usr/share/config /var/qmail/control"
CONFIG_PROTECT_MASK="/etc/gconf /etc/terminfo /etc/env.d"
CXXFLAGS="-march=athlon-xp -fomit-frame-pointer -O2 -falign-jumps -pipe"
DISTDIR="/usr/portage/distfiles"
FEATURES="autoaddcvs buildpkg ccache sandbox sfperms userpriv"
GENTOO_MIRRORS="ftp://ftp.ucsb.edu/pub/mirrors/linux/gentoo/  http://128.213.5.34/gentoo/"
MAKEOPTS="-j3 "
PKGDIR="/usr/portage/packages"
PORTAGE_TMPDIR="/home/tmp"
PORTDIR="/usr/portage"
PORTDIR_OVERLAY="/usr/include/portage"
SYNC="rsync://rsync.gentoo.org/gentoo-portage"
USE="X arts async autofs avi berkdb bitmap-fonts cdr chroot crypt cups dillo doc encode esd foomaticdb freetype gdbm gif gnome gpm gtk gtk2 gvim hpijs imagemagick imlib jpeg kde libg++ libwww mad mikmod motif mozilla mp3 mpeg mpeg4 mplayer ncurses nptl oggvorbis oss pam pdflib perl png python qt quicktime readline samba sdl slang snmp spell ssl svga tcpd tiff truetype type1 unicode x86 xml2 xmms xprint xv zlib" 
Comment 7 Hiel Van Campen 2004-09-29 09:22:38 UTC
See this thread for why mirrorselect is having troubles. http://forums.gentoo.org/viewtopic.php?t=229921&start=0&postdays=0&postorder=asc&highlight= 
To sum it up /dev/hull is being changed to a regular file while there bootstraping and glibc breaks. Mirrorselect tries to dump stuff to /dev/null and /dev/null aint  /dev/null any longer.
Comment 8 SpanKY gentoo-dev 2004-10-02 14:15:11 UTC
*** Bug 66115 has been marked as a duplicate of this bug. ***
Comment 9 Martin von Gagern 2004-10-09 09:57:01 UTC
This seems to be a duplicate of bug 61496
See also bug 65498
Comment 10 John Mylchreest (RETIRED) gentoo-dev 2004-10-09 11:21:20 UTC
see bug 61496
Comment 11 Anthony Metcalf 2004-10-12 03:33:14 UTC
I just ran 

mirrorselect -a -s3 -b10 -D -o >> /etc/make.conf 

as my downloads seemed abnormally slow.

I got the problem documented here. 

This is with mirrorselect-0.84 and only occured when I added the -D flag.

I added this to this bug as the symptoms seem so similar. Hope that isn't a problem.