the delta file and also the .lzma tarball are missing
huh? % wget distfiles.gentoo.org/snapshots/deltas/snapshot-20110110-20110111.patch.bz2 --2011-01-18 00:07:15-- http://distfiles.gentoo.org/snapshots/deltas/snapshot-20110110-20110111.patch.bz2 Resolving distfiles.gentoo.org... 64.50.236.52, 137.226.34.42, 140.211.166.134, ... Connecting to distfiles.gentoo.org|64.50.236.52|:80... connected. HTTP request sent, awaiting response... 200 OK Length: 157050 (153K) [application/x-bzip2] Saving to: `snapshot-20110110-20110111.patch.bz2' 100%[======================================>] 157,050 840K/s in 0.2s 2011-01-18 00:07:16 (840 KB/s) - `snapshot-20110110-20110111.patch.bz2' saved [157050/157050]
huh! The file was created on the 14th, two days after the bug was reported. And still you broke the upgrade path because all older delta files were removed. This is what bug #351822 is about.
(In reply to comment #2) > huh! The file was created on the 14th, two days after the bug was reported. > > And still you broke the upgrade path because all older delta files were > removed. This is what bug #351822 is about. > Yea, I see that the older files were removed. Not sure what happened there...
*** Bug 351822 has been marked as a duplicate of this bug. ***
We're not sure what happened. Restored that directory from another machine, hopefully it doesn't happen again. =/ Should be hitting mirrors soon.
Well, the files are there now, but emerge-delta-webrsync still doesn't work cleanly: after downloading the patches, it prints the following: verbosity level(1) patch_type=8 patch_type=8 patch_type=8 patch_type=8 patch_type=8 patch_type=8 patch_type=8 patch_type=8 patch_type=8 patch_type=8 patch_type=8 disabling bufferless, patch_count(11) == 1 || forced_reorder(1) size1=297932800, size2=297912320 reconstruction return=0, commands=8756 result was 8756 commands versions size is 297912320 index_size = 4378 size1=297912320, size2=297973760 reconstruction return=0, commands=15187 result was 15187 commands versions size is 297973760 index_size = 7593 size1=297973760, size2=296314880 reconstruction return=0, commands=40787 result was 40787 commands versions size is 296314880 index_size = 20393 size1=296314880, size2=296273920 reconstruction return=0, commands=52019 result was 52019 commands versions size is 296273920 index_size = 26009 size1=296273920, size2=296478720 reconstruction return=0, commands=64993 result was 64993 commands versions size is 296478720 index_size = 32496 size1=296478720, size2=296785920 reconstruction return=0, commands=79099 result was 79099 commands versions size is 296785920 index_size = 39549 size1=296785920, size2=295802880 reconstruction return=0, commands=91002 result was 91002 commands versions size is 295802880 index_size = 45501 size1=295802880, size2=296007680 reconstruction return=0, commands=102059 result was 102059 commands versions size is 296007680 index_size = 51029 size1=296007680, size2=296079360 reconstruction return=0, commands=110748 result was 110748 commands versions size is 296079360 index_size = 55374 size1=296079360, size2=295598080 reconstruction return=0, commands=119557 result was 119557 commands versions size is 295598080 index_size = 59778 size1=295680000, size2=295874560 /usr/bin/emerge-delta-webrsync: line 440: 1570 Segmentation fault patcher -v "${dfile}" ${patches} "${TEMPDIR}/portage-${final_date}.tar" reconstruction failed (contact the author with the error from the reconstructor please) Fetching most recent snapshot And after that, it proceeds with downloading the full portage tree. Kernel log contains the following: Jan 19 11:40:49 aurora kernel: patcher[1570]: segfault at 9a007d82ea ip 00007f42afa3e16e sp 00007fff9cabad80 error 4 in libdiffball.so.0.0.0[7f42afa32000+13000] The same thing happens on two machines that have portage-20110107.tar.bz2 in their distfiles. I suspect some of patch files are incorrect (like, generated from a different set of snapshots). My portage-20110107.tar.bz2's MD5 is 1d728250a3e240f456d83a37438c722e. On the other hand, if I allow emerge-delta-webrsync to continue, it will happily fetch the latest snapshot and use it. So having *some* patches (even if they are incorrect) is kind of a work-around for the original problem. :)
I feared that. The files I restored were generated on another machine. The new machine is faster and hence has slightly different timestamps (minutes or seconds different). This is probably the source of the md5sum difference. I'm not inclined to spend a ton of time on this. Sorry for the inconvenience. =/
Then I suggest replacing the "broken" patch files with something that the patcher program *won't* recognize and thus won't use, like, the files with text "No patch available", maybe with the link to this bug :) . And regenerate md5 files, of course. This way, patcher will exit cleanly instead of segfaulting (which is important to prevent, IMO), and emerge-delta-webrsync will proceed with full snapshot download. Also, the "won't fix" resolution would probably be more appropriate for this bug, given the circumstances.
Pavel, excuse my ignorance. What happens if I remove all the deltas and let the daily patches "start over" ?
And now, I've just restored the deltas from a backup. Life should be good again, please test and report back. (Still don't know why they disappeared in the first place)
Hm, not sure I understand the second part of the question... If patches (or their MD5) are removed from the mirror and are not present locally (specifically, if there is no patch available that would update the local portage snapshot - the file portage-YYYYMMDD.tar.bz2 in distfiles - to the next date), emerge-delta-webrsync will think that the local snapshot is the latest available one and will either exit right away or sync the portage tree with the local snapshot, depending on whether the -u option was specified. If patches are removed locally - see my comment #6 on that happens (in case the local portage snapshot is older than 2011-01-09 or something). UPDATE: With the patches properly restored from the backup, all should be ok as long as I delete the "bad" patches that were already downloaded.
Ok. I see what you mean. We have an upgrade path now in place for move the hosts that generates the snapshots. (Also, the files get mirrored at XX:05, so it could take a few hours to fully propagate if you are testing it)
Ok, now I was able to successfully complete sync (after deleting local copies of patches, of course). Life is good again, as promised. :)
