emerge --sync not using round-robin--it always chooses the same server. For example: chronos ~ # emerge --sync >>> Starting rsync with rsync://209.177.148.226/gentoo-portage... >>> Checking server timestamp ... Welcome to cockatoo.gentoo.org Server Address : 209.117.148.226 Contact Name : mirror-admin@gentoo.org <snip> chronos ~ # emerge --sync >>> Starting rsync with rsync://209.177.148.226/gentoo-portage... >>> Checking server timestamp ... Welcome to cockatoo.gentoo.org Server Address : 209.117.148.226 Contact Name : mirror-admin@gentoo.org <snip> If I do it 100 times, it will select the same address everytime. I use a local nameserver and if I do "rndc flush" in-between, it still does the same thing. I also have tried using other nameservers, but again, it always selects that address. I have tried setting the SYNC to other round-robin ones (rsync.namerica.gentoo.org, etc.), but it does the same thing (it picks the same one from the list--with rsync.namerica.gentoo.org it always picks rsync10.us.gentoo.org).
Here's the usual emerge --info (for two machines): chronos ~ # emerge --info Portage 2.1.2-r5 (default-linux/amd64/2006.1, gcc-4.1.1, glibc-2.4-r4, 2.6.18-gentoo-r6 x86_64) ================================================================= System uname: 2.6.18-gentoo-r6 x86_64 Intel(R) Core(TM)2 CPU 6700 @ 2.66GHz Gentoo Base System version 1.12.6 Timestamp of tree: Mon, 29 Jan 2007 06:50:01 +0000 dev-lang/python: 2.4.3-r4 dev-python/pycrypto: 2.0.1-r5 sys-apps/sandbox: 1.2.17 sys-devel/autoconf: 2.13, 2.61 sys-devel/automake: 1.4_p6, 1.5, 1.6.3, 1.7.9-r1, 1.8.5-r3, 1.9.6-r2, 1.10 sys-devel/binutils: 2.16.1-r3 sys-devel/gcc-config: 1.3.14 sys-devel/libtool: 1.5.22 virtual/os-headers: 2.6.19.2-r2 ACCEPT_KEYWORDS="amd64" AUTOCLEAN="yes" CBUILD="x86_64-pc-linux-gnu" CFLAGS="-march=nocona -O2 -pipe" CHOST="x86_64-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/kde/3.5/env /usr/kde/3.5/share/config /usr/kde/3.5/shutdown /usr/share/X11/xkb /usr/share/config /var/bind" CONFIG_PROTECT_MASK="/etc/env.d /etc/gconf /etc/revdep-rebuild /etc/terminfo /etc/texmf/web2c" CXXFLAGS="-march=nocona -O2 -pipe" DISTDIR="/usr/portage/distfiles" FEATURES="autoconfig distlocks metadata-transfer sandbox sfperms strict" GENTOO_MIRRORS="ftp://mirror.datapipe.net/gentoo http://gentoo.chem.wisc.edu/gentoo/ ftp://ftp.tu-clausthal.de/pub/linux/gentoo/" MAKEOPTS="-j3" PKGDIR="/usr/portage/packages" PORTAGE_RSYNC_EXTRA_OPTS="--exclude-from=/etc/portage/rsync_excludes" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --delete-after --stats --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages" PORTAGE_TMPDIR="/var/tmp" PORTDIR="/usr/portage" SYNC="rsync://rsync.namerica.gentoo.org/gentoo-portage" USE="X aac alsa amd64 amr arts bash-completion berkdb bitmap-fonts bzip2 cairo cli cracklib crypt cups curl dlloader dri dvd dvdread encode fam ffmpeg flac fortran gcc64 gdbm gif gpm iconv ieee1394 imlib ipv6 isdnlog jpeg kde kdeenablefinal libg++ logitech-mouse mad matroska mng mp3 ncurses nls nptl nptlonly ogg opengl pam pcre perl png ppds pppd python qt4 readline reflection rtc ruby sdl session smp spell spl ssl symlink tcpd tetex threads truetype truetype-fonts type1-fonts udev unicode usb vorbis x264 xorg xv xvid zlib" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mulaw multi null plug rate route share shm softvol" ELIBC="glibc" INPUT_DEVICES="evdev keyboard mouse" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" USERLAND="GNU" VIDEO_CARDS="dummy fbdev nvidia vesa vga" Unset: CTARGET, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LANG, LC_ALL, LDFLAGS, LINGUAS, PORTDIR_OVERLAY saturn ~ # emerge --info Portage 2.1.1-r2 (default-linux/x86/2006.1/server, gcc-4.1.1, glibc-2.4-r4, 2.6.18-gentoo-r6 i686) ================================================================= System uname: 2.6.18-gentoo-r6 i686 Intel(R) Xeon(TM) CPU 3.00GHz Gentoo Base System version 1.12.6 Last Sync: Sun, 28 Jan 2007 07:00:01 +0000 app-admin/eselect-compiler: [Not Present] dev-java/java-config: 1.3.7, 2.0.31 dev-lang/python: 2.4.3-r4 dev-python/pycrypto: 2.0.1-r5 dev-util/ccache: [Not Present] dev-util/confcache: [Not Present] sys-apps/sandbox: 1.2.17 sys-devel/autoconf: 2.13, 2.61 sys-devel/automake: 1.4_p6, 1.5, 1.6.3, 1.7.9-r1, 1.8.5-r3, 1.9.6-r2, 1.10 sys-devel/binutils: 2.16.1-r3 sys-devel/gcc-config: 1.3.14 sys-devel/libtool: 1.5.22 virtual/os-headers: 2.6.17-r2 ACCEPT_KEYWORDS="x86" AUTOCLEAN="yes" CBUILD="i686-pc-linux-gnu" CFLAGS="-march=pentium4 -O2 -pipe" CHOST="i686-pc-linux-gnu" CONFIG_PROTECT="/etc" CONFIG_PROTECT_MASK="/etc/env.d /etc/env.d/java/ /etc/gconf /etc/java-config/vms/ /etc/revdep-rebuild /etc/terminfo" CXXFLAGS="-O2 -mcpu=i686 -pipe" DISTDIR="/usr/portage/distfiles" FEATURES="autoconfig distlocks metadata-transfer sandbox sfperms strict" GENTOO_MIRRORS="http://distfiles.gentoo.org http://distro.ibiblio.org/pub/linux/distributions/gentoo" MAKEOPTS="-j5" PKGDIR="/usr/portage/packages" PORTAGE_RSYNC_EXTRA_OPTS="--exclude-from=/etc/portage/rsync_excludes" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --delete-after --stats --timeout=180 --exclude='/distfiles' --exclude='/local' --exclude='/packages'" PORTAGE_TMPDIR="/var/tmp" PORTDIR="/usr/portage" SYNC="rsync://rsync.gentoo.org/gentoo-portage" USE="x86 alsa_cards_ali5451 alsa_cards_als4000 alsa_cards_atiixp alsa_cards_atiixp-modem alsa_cards_bt87x alsa_cards_ca0106 alsa_cards_cmipci alsa_cards_emu10k1x alsa_cards_ens1370 alsa_cards_ens1371 alsa_cards_es1938 alsa_cards_es1968 alsa_cards_fm801 alsa_cards_hda-intel alsa_cards_intel8x0 alsa_cards_intel8x0m alsa_cards_maestro3 alsa_cards_trident alsa_cards_usb-audio alsa_cards_via82xx alsa_cards_via82xx-modem alsa_cards_ymfpci alsa_pcm_plugins_adpcm alsa_pcm_plugins_alaw alsa_pcm_plugins_asym alsa_pcm_plugins_copy alsa_pcm_plugins_dmix alsa_pcm_plugins_dshare alsa_pcm_plugins_dsnoop alsa_pcm_plugins_empty alsa_pcm_plugins_extplug alsa_pcm_plugins_file alsa_pcm_plugins_hooks alsa_pcm_plugins_iec958 alsa_pcm_plugins_ioplug alsa_pcm_plugins_ladspa alsa_pcm_plugins_lfloat alsa_pcm_plugins_linear alsa_pcm_plugins_meter alsa_pcm_plugins_mulaw alsa_pcm_plugins_multi alsa_pcm_plugins_null alsa_pcm_plugins_plug alsa_pcm_plugins_rate alsa_pcm_plugins_route alsa_pcm_plugins_share alsa_pcm_plugins_shm alsa_pcm_plugins_softvol apache2 bash-completion berkdb bzip2 clamd cli cracklib crypt cups curlwrappers dlloader dri elibc_glibc gd gdbm gif gpm iconv input_devices_evdev input_devices_keyboard input_devices_mouse isdnlog jpeg kernel_linux lcd_devices_bayrad lcd_devices_cfontz lcd_devices_cfontz633 lcd_devices_glk lcd_devices_hd44780 lcd_devices_lb216 lcd_devices_lcdm001 lcd_devices_mtxorb lcd_devices_ncurses lcd_devices_text libg++ mailwrapper ncurses nls no-old-linux nptl nptlonly pam pcre pear perl php png postgres ppds pppd python readline reflection session snmp spl sse2 ssl symlink threads udev unicode userland_GNU video_cards_apm video_cards_ark video_cards_ati video_cards_chips video_cards_cirrus video_cards_cyrix video_cards_dummy video_cards_fbdev video_cards_glint video_cards_i128 video_cards_i740 video_cards_i810 video_cards_imstt video_cards_mga video_cards_neomagic video_cards_nsc video_cards_nv video_cards_rendition video_cards_s3 video_cards_s3virge video_cards_savage video_cards_siliconmotion video_cards_sis video_cards_sisusb video_cards_tdfx video_cards_tga video_cards_trident video_cards_tseng video_cards_v4l video_cards_vesa video_cards_vga video_cards_via video_cards_vmware video_cards_voodoo wddx xml xorg zlib" Unset: CTARGET, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LANG, LC_ALL, LDFLAGS, LINGUAS, PORTDIR_OVERLAY
$ date ; rsync rsync://rsync.us.gentoo.org:: Mon Jan 29 19:27:01 EST 2007 Welcome to gentoo.boxacle.net/rsync23.us.gentoo.org Server Address : 209.189.242.21 Contact Name : admin@boxacle.net Hardware : 1 x AMD Opteron 244 (1.8 GHz) , 1 GB RAM Network: : 10 Mb/s Full Duplex Connections : 50 concurrent Location : Bryan, TX, USA Please note: common gentoo-netiquette says you should not sync more than once a day. Users who abuse the rsync.gentoo.org rotation may be added to a temporary ban list. gentoo-portage Gentoo Linux Portage tree mirror $ date ; rsync rsync://rsync.us.gentoo.org:: Mon Jan 29 19:27:05 EST 2007 Gentoo Portage/CPAN rsync mirror Server: gentoo.llarian.net & cpan.llarian.net IP(s): 209.221.142.124 & 2001:470:1f01:164::4 Server Specs: Dual PIII-1Ghz, 1GB RAM Bandwidth: 100Mbps via multiple carriers User Limit: 30 Location: Seattle, WA, USA Admin Contact: Dylan Vanderhoof <llarian@llarian.net> gentoo-x86-portage Gentoo Linux Portage tree gentoo-portage Gentoo Linux Portage tree mirror gentoo Gentoo download mirror CPAN Comprehensive Perl Archive Network $ date ; rsync rsync://rsync.us.gentoo.org:: Mon Jan 29 19:27:07 EST 2007 gentoo-portage Gentoo Portage tree $ date ; rsync rsync://rsync.us.gentoo.org:: Mon Jan 29 19:27:11 EST 2007 (cut) Through contributions from GT-RNOC, we have also expanded network capacity. All servers now have gigabit links to the campus backbone as well as a private gigabit network for NFS. I'm thinking your ISP is doing some agressive DNS caching
Well, as said above, will happen with DNS caching.
I guess you didn't bother to read my post in its entirety. I run my own nameserver locally, and there is nothing "aggressive" about the caching--in fact, as I mentioned earlier, I did "rndc flush" (flushed the cache) in-between runs and used several other nameservers (my ISP's, my work's, and my former college's), and still had the same problem. Additionally, the machines I posted the "emerge --info" from are on separate networks (literally 5 miles away from each other using different ISPs), running different versions of portage, using different name servers, and both exhibit the same behavior. Further, if I use dig the round-robin works fine. So, DNS caching is not the problem. At this point, I'm positive it has something to do with getaddrinfo(). I wrote a little test program in python: import socket for res in socket.getaddrinfo('rsync.namerica.gentoo.org', None, socket.AF_UNSPEC, socket.SOCK_STREAM, 0, socket.AI_PASSIVE): print res[4][0] I ran it quite a few times, and noticed that the first two IP addresses were always the same (when using rsync.gentoo.org only the first one stayed the same--so it must be dependant on the total size of the result set). However, the remaining IP addresses rotated as they should.
(In reply to comment #4) > I guess you didn't bother to read my post in its entirety. I run my own > nameserver locally, and there is nothing "aggressive" about the caching--in > fact, as I mentioned earlier, I did "rndc flush" (flushed the cache) in-between > runs and used several other nameservers (my ISP's, my work's, and my former > college's), and still had the same problem. > > Additionally, the machines I posted the "emerge --info" from are on separate > networks (literally 5 miles away from each other using different ISPs), running > different versions of portage, using different name servers, and both exhibit > the same behavior. Further, if I use dig the round-robin works fine. > > So, DNS caching is not the problem. > > At this point, I'm positive it has something to do with getaddrinfo(). I wrote > a little test program in python: > > import socket > > for res in socket.getaddrinfo('rsync.namerica.gentoo.org', None, > socket.AF_UNSPEC, socket.SOCK_STREAM, 0, socket.AI_PASSIVE): > print res[4][0] > > I ran it quite a few times, and noticed that the first two IP addresses were > always the same (when using rsync.gentoo.org only the first one stayed the > same--so it must be dependant on the total size of the result set). However, > the remaining IP addresses rotated as they should. > I'll apply a patch for this later today.
Created attachment 108599 [details, diff] shuffle the ip addresses returned from socket.getaddrinfo() This patch should fix it. It's in svn r5833.
This has been released in 2.1.2-r6.
