For some days now emerge-delta-webrsync fails on all my computers: # emerge-delta-webrsync Looking for available base versions for a delta Checking digest ... fetching patches Fetching file snapshot-20160203-20160204.patch.bz2 ... Looking for available base versions for a delta Checking digest ... fetching patches Fetching file snapshot-20160203-20160204.patch.bz2.md5sum ... --2016-02-05 11:17:37-- http://distfiles.gentoo.org/snapshots/deltas/snapshot-20160203-20160204.patch.bz2.md5sum Resolving distfiles.gentoo.org... 64.50.233.100, 137.226.34.46, 140.211.166.134, ... Connecting to distfiles.gentoo.org|64.50.233.100|:80... connected. HTTP request sent, awaiting response... 200 OK Length: 71 [application/x-bzip2] Saving to: ‘/gentoo/distfiles/snapshot-20160203-20160204.patch.bz2.md5sum’ /gentoo/distfiles/snapshot 100%[=======================================>] 71 --.-KB/s in 0s 2016-02-05 11:17:38 (13.6 MB/s) - ‘/gentoo/distfiles/snapshot-20160203-20160204.patch.bz2.md5sum’ saved [71/71] Fetching file snapshot-20160203-20160204.patch.bz2 ... --2016-02-05 11:17:38-- http://distfiles.gentoo.org/snapshots/deltas/snapshot-20160203-20160204.patch.bz2 Resolving distfiles.gentoo.org... 64.50.233.100, 137.226.34.46, 140.211.166.134, ... Connecting to distfiles.gentoo.org|64.50.233.100|:80... connected. HTTP request sent, awaiting response... 200 OK Length: 238216 (233K) [application/x-bzip2] Saving to: ‘/gentoo/distfiles/snapshot-20160203-20160204.patch.bz2’ /gentoo/distfiles/snapshot 100%[=======================================>] 232.63K 520KB/s in 0.4s 2016-02-05 11:17:38 (520 KB/s) - ‘/gentoo/distfiles/snapshot-20160203-20160204.patch.bz2’ saved [238216/238216] Checking digest ... Fetching file snapshot-20160204-20160205.patch.bz2 ... --2016-02-05 11:17:38-- http://distfiles.gentoo.org/snapshots/deltas/snapshot-20160204-20160205.patch.bz2 Resolving distfiles.gentoo.org... 64.50.233.100, 137.226.34.46, 140.211.166.134, ... Connecting to distfiles.gentoo.org|64.50.233.100|:80... connected. HTTP request sent, awaiting response... 404 Not Found 2016-02-05 11:17:38 ERROR 404: Not Found. failed fetching snapshot-20160204-20160205.patch.bz2 Fetching file portage-20160204.tar.bz2.md5sum ... --2016-02-05 11:17:38-- http://distfiles.gentoo.org/snapshots/portage-20160204.tar.bz2.md5sum Resolving distfiles.gentoo.org... 64.50.233.100, 137.226.34.46, 140.211.166.134, ... Connecting to distfiles.gentoo.org|64.50.233.100|:80... connected. HTTP request sent, awaiting response... 200 OK Length: 59 [application/x-bzip2] Saving to: ‘/gentoo/distfiles/portage-20160204.tar.bz2.md5sum’ /gentoo/distfiles/portage- 100%[=======================================>] 59 --.-KB/s in 0s 2016-02-05 11:17:39 (12.4 MB/s) - ‘/gentoo/distfiles/portage-20160204.tar.bz2.md5sum’ saved [59/59] Fetching file portage-20160204.tar.bz2.umd5sum ... --2016-02-05 11:17:39-- http://distfiles.gentoo.org/snapshots/portage-20160204.tar.bz2.umd5sum Resolving distfiles.gentoo.org... 64.50.233.100, 137.226.34.46, 140.211.166.134, ... Connecting to distfiles.gentoo.org|64.50.233.100|:80... connected. HTTP request sent, awaiting response... 200 OK Length: 55 [application/x-bzip2] Saving to: ‘/gentoo/distfiles/portage-20160204.tar.bz2.umd5sum’ /gentoo/distfiles/portage- 100%[=======================================>] 55 --.-KB/s in 0s 2016-02-05 11:17:39 (11.6 MB/s) - ‘/gentoo/distfiles/portage-20160204.tar.bz2.umd5sum’ saved [55/55] Fetching file portage-20160204.tar.bz2.gpgsig ... --2016-02-05 11:17:39-- http://distfiles.gentoo.org/snapshots/portage-20160204.tar.bz2.gpgsig Resolving distfiles.gentoo.org... 64.50.233.100, 137.226.34.46, 140.211.166.134, ... Connecting to distfiles.gentoo.org|64.50.233.100|:80... connected. HTTP request sent, awaiting response... 200 OK Length: 819 [application/x-bzip2] Saving to: ‘/gentoo/distfiles/portage-20160204.tar.bz2.gpgsig’ /gentoo/distfiles/portage- 100%[=======================================>] 819 --.-KB/s in 0s 2016-02-05 11:17:39 (164 MB/s) - ‘/gentoo/distfiles/portage-20160204.tar.bz2.gpgsig’ saved [819/819] verbosity level(1) patch_type=8 disabling bufferless, patch_count(1) == 1 || forced_reorder(1) size1=560670720, size2=560773120 reconstruction return=0, commands=98249 result was 98249 commands versions size is 560773120 applied 1 patches reordering commands? 1 reconstructing target file based off of dcbuff commands... collapsing processing src 0: 55174 commands. processing src 1: 43075 commands. reconstruction completed successfully verifying uncompressed md5 Checking digest ... recompressing ... /tmp/portage64/portage/delta-webrsync-8MDS2c/portage-20160204.tar: 7.437:1, 1.076 bits/byte, 86.55% saved, 560773120 in, 75398994 out. Checking signature ... gpg: WARNING: unsafe permissions on homedir `/etc/portage/gnupg' gpg: Signature made Fri 05 Feb 2016 12:56:56 AM UTC using RSA key ID C9189250 gpg: BAD signature from "Gentoo Portage Snapshot Signing Key (Automated Signing Key)" [unknown] As can be seen the uncompressed md5 is ok, but not the (re-)compressed md5: # md5sum -c /gentoo/distfiles/portage-20160204.tar.bz2.md5sum portage-20160204.tar.bz2: FAILED md5sum: WARNING: 1 computed checksum did NOT match As emerge-delta-webrsync and bzip2 have not been changed in portage for quite some time I would guess that snapshot compression on the gentoo server has changed (maybe parallel bzip2?).
Try: gpg --refresh-keys
This is not bug 570734. My keys are up to date: # gpg --homedir /etc/portage/gnupg --list-keys /etc/portage/gnupg/pubring.gpg ------------------------------ pub 4096R/96D8BF6D 2011-11-25 [expires: 2016-07-01] uid [ unknown] Gentoo Portage Snapshot Signing Key (Automated Signing Key) sub 4096R/C9189250 2011-11-25 [expires: 2016-07-01]
This is not a key issue; the tarball emerge-delta-webrsync produces is different from the original. md5sums: 2852f5b070e5db382c22bdef98350ffa /var/tmp/portage/delta-webrsync-F3zQ9V/portage-20160205.tar.bz2 2d54187ce61a7599f7a0017676b5f36a portage-20160205.tar.bz2 After bunzip2: 0d4e56b12f1b9cfae871d7a5b59c5a39 portage-20160205-diff.tar 0d4e56b12f1b9cfae871d7a5b59c5a39 portage-20160205-orig.tar sha256sums just in case: 29d3f073e6bc1a1dd2c5f7d453dedab5c9183b8043a055ce438a4e0d3ad70916 portage-20160205-diff.tar 29d3f073e6bc1a1dd2c5f7d453dedab5c9183b8043a055ce438a4e0d3ad70916 portage-20160205-orig.tar bzip2 -vk9 (as emerge-delta-webrsync uses) yeilds the same file (expected): 2852f5b070e5db382c22bdef98350ffa portage-20160205-diff.tar.bz2 lbzip2 -k9 yeilds: 2d54187ce61a7599f7a0017676b5f36a portage-20160205-diff.tar.bz2 Which matches the bz2 in the repos. So, it appears emerge-delta-webrsync users now needs to start using lbzip2.
Relying on specific output from recompression is a bad idea. I think we should start providing a GPG signature of the uncompressed tarball, and emerge-delta-webrsync should verify that instead.
I've updated the scripts now, so hopefully the next snapshot will work again. Please let me know if that's the case but don't close the bug since we really need to update the way e-d-w verifies tarballs.
emerge-delta-webrsync could successfully create portage-20160208.tar.bz2 from portage-20160207.tar.bz2. Thanks for your effort, Michał.
(In reply to Michał Górny from comment #5) > I've updated the scripts now, so hopefully the next snapshot will work > again. Please let me know if that's the case but don't close the bug since > we really need to update the way e-d-w verifies tarballs. Heh, Zac mentioned this a while ago in bug 286373 too.