Hi, I have a mirror of all the gentoo tree, and have been syncing it from rsync://www.iblibio.org/gentoo, for quite a while. I think the problem arises when rsync requests the server for beta26_full.zip, from what I understand of the log file. Reproducible: Always Steps to Reproduce: 1. rsync -rlptzD -vvvv --stats --progress --delete --delete-after --timeout=6000 rsync://www.ibiblio.org/gentoo /gentoo/ Actual Results: ----- 1 ----- opening tcp connection to www.ibiblio.org port 873 (motd) receiving file list ... recv_file_name(.) recv_file_name(distfiles) recv_file_name(distfiles/CUnit-1.0-6.tar.gz) recv_file_name(distfiles/Sjeng-Free-11.2.tar.gz) recv_file_name(distfiles/BeOS-4ever) . (lots of recv_file_name()) . recv_file_name(experimental/mips/stages/stage1-mips-1.4-beta.tbz2) recv_file_name(experimental/mips/stages/stage2-mips-1.4-beta.tbz2) recv_file_name(experimental/mips/stages/stage3-mips-1.4-beta.tbz2) received 9536 names [62] i=0 <NULL> . mode=042755 len=4096 [62] i=1 <NULL> distfiles mode=040755 len=241664 [62] i=2 distfiles 0.60.1-0.61.1-pr1.diff.bz2 mode=0100644 len=797886 [62] i=3 distfiles 0.61.1-pr1-0.61.1-pr2.diff.bz2 mode=0100644 len=6448 . (more indexing...) . [62] i=9533 snapshots portage-20030220.tar.bz2 mode=0100644 len=6712527 [62] i=9534 snapshots portage-20030221.tar.bz2 mode=0100644 len=6725619 [62] i=9535 snapshots portage-20030222.tar.bz2 mode=0100644 len=6713676 9536 files to consider recv_file_list done get_local_name count=9536 /gentoo/ generator starting pid=62 count=9536 delta transmission enabled recv_generator(.,0) recv_generator(distfiles,1) set modtime of distfiles to (1046142838) Tue Feb 25 03:13:58 2003 distfiles/ recv_generator(distfiles/0.60.1-0.61.1-pr1.diff.bz2,2) distfiles/0.60.1-0.61.1-pr1.diff.bz2 is uptodate recv_generator(distfiles/0.61.1-pr1-0.61.1-pr2.diff.bz2,3) distfiles/0.61.1-pr1-0.61.1-pr2.diff.bz2 is uptodate . (lots of recv_generator(), but all uptodate, till beta26_full.zip) . recv_generator(distfiles/beos-1.2.tar.gz,1168) distfiles/beos-1.2.tar.gz is uptodate recv_generator(distfiles/beos.zip,1169) distfiles/beos.zip is uptodate recv_generator(distfiles/beta26_full.zip,1170) gen mapped distfiles/beta26_full.zip of size 250942395 generating and sending sums for 1170 count=15317 rem=5051 n=16384 flength=250942395 recv_files(9536) starting chunk[0] offset=0 len=16384 sum1=3d910c79 chunk[1] offset=16384 len=16384 sum1=1c89f687 chunk[2] offset=32768 len=16384 sum1=b501b500 chunk[3] offset=49152 len=16384 sum1=ec6b036a chunk[4] offset=65536 len=16384 sum1=e10ef73d chunk[5] offset=81920 len=16384 sum1=3df1cb5e . (so the client starts to 'generating and sending sums for 1170') . chunk[15314] offset=250904576 len=16384 sum1=5d95ea15 chunk[15315] offset=250920960 len=16384 sum1=e987d8a3 chunk[15316] offset=250937344 len=5051 sum1=bb426785 recv_generator(distfiles/bfm-0.5.1.tar.bz2,1171) gen mapped distfiles/bfm-0.5.1.tar.bz2 of size 44286 generating and sending sums for 1171 count=64 rem=186 n=700 flength=44286 chunk[0] offset=0 len=700 sum1=6b3f09e2 chunk[1] offset=700 len=700 sum1=6053e7d6 chunk[2] offset=1400 len=700 sum1=1fd0fb1e . (and does so for quite a few files, "grep "generating and sending sums" rsync.output |wc -l" retuns 13) . chunk[61] offset=42700 len=700 sum1=48dcfebe chunk[62] offset=43400 len=700 sum1=4fc0f4ec chunk[63] offset=44100 len=186 sum1=2f84fe1e recv_generator(distfiles/bglibs-1.001.tar.gz,1172) gen mapped distfiles/bglibs-1.001.tar.gz of size 62354 generating and sending sums for 1172 count=90 rem=54 n=700 flength=62354 chunk[0] offset=0 len=700 sum1=e7a5089a chunk[1] offset=700 len=700 sum1=94ad03f4 chunk[2] offset=1400 len=700 sum1=ecb9f5bf . (the other file's chunks, till this last one:) . recv_generator(distfiles/bigloo2.4c.tar.gz,1182) gen mapped distfiles/bigloo2.4c.tar.gz of size 7791033 generating and sending sums for 1182 count=10145 rem=441 n=768 flength=7791033 chunk[0] offset=0 len=768 sum1=6a57fe36 chunk[1] offset=768 len=768 sum1=35b2f88d chunk[2] offset=1536 len=768 sum1=7722ef5f . . . chunk[9329] offset=7164672 len=768 sum1=2cb704ff chunk[9330] offset=7165440 len=768 sum1=9af9f8fc chunk[9331] offset=7166208 len=768 sum1=ca67f2c2 recv_files(distfiles/beta26_full.zip) recv mapped distfiles/beta26_full.zip of size 250942395 distfiles/beta26_full.zip rsync: read error: Connection reset by peer _exit_cleanup(code=12, file=io.c, line=177): entered rsync error: error in rsync protocol data stream (code 12) at io.c(177) _exit_cleanup(code=12, file=io.c, line=177): about to call exit(12) rsync: connection unexpectedly closed (284099 bytes read so far) _exit_cleanup(code=12, file=io.c, line=165): entered rsync error: error in rsync protocol data stream (code 12) at io.c(165) _exit_cleanup(code=12, file=io.c, line=165): about to call exit(12) (on of the 2 recv_files() call was for distfiles/beta26_full.zip, and after it has been requested the server returned 'Connection reset by peer'. The other call was 'recv_files(9536) starting' and you can search it above) ----- /1 ----- Expected Results: had synced the tree. - I tried to sync from rsync://ftp.snt.utwente.nl/gentoo too, but the problem remains. - From my log files, the error has been first return on 'Mon Feb 24 20:36:00 WET 2003', and repetly does till this moment. I think that either this happened because of a timeout, or rsync is not prepared to sync 250MBs files. I have the complete log file, but I don't think that I should post it, because it's 2.8MB uncompressed and 464KB on the compressed format. I'm going to erase beta26_full.zip and sync the tree. Let's see if this does the trick. I would appreciate any suggestion,
did a spot check of 3 other distfile mirrors and all had the correct zip file, so rsync seems to be working on ibiblio's side. I'm assuming your mirror is in portugal (is it public, btw?) so you may be running into latency and/or timeout issues. Suggestions include deleting the zip file on your side of things (as you've already mentioned) and downloading the file manually so rsync will hopefully skip it. we are working on creating a private "master" distfile mirror that will be password-protected and used to sync all other distfile mirrors from. (it won't be an ibiblio server) so hopefully this will resolve the problem. marking this bug as closed -- worksforme with the understanding that the master distfile mirror mentioned above may help your problem.
The mirror is portuguese and semi-public. My University doesn't have the necessary bandwidth to make it public. It's mentioned on a post[1] in the Portuguese Forum[2], and advertised on #gentoo-pt's channel topic. I moved the file from the distfiles directory, and I think it's working. At least it's still updating (low bandwidth). Thanks for the reply, Ricardo Cordeiro :) [1]http://forums.gentoo.org/viewtopic.php?p=190390#190390 [2]http://forums.gentoo.org/viewforum.php?f=39
I turned off the -z flag, and everything worked fine. I hope nobody is on my previous situation right now.
I'm having the exact same problem, I would imagine the problem comes from trying to compress that 240MB file, it takes too long and the connection times out. It worked for me too when I took off the compress flag.
