Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 573908 - emerge-delta-webrsync / infra: need to add GPG signature of uncompressed tarball rather than compressed one
Summary: emerge-delta-webrsync / infra: need to add GPG signature of uncompressed tarb...
Status: CONFIRMED
Alias: None
Product: Portage Development
Classification: Unclassified
Component: Tools (show other bugs)
Hardware: All Linux
: Normal normal (vote)
Assignee: Portage Tools Team
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2016-02-05 11:19 UTC by sf
Modified: 2023-08-19 15:55 UTC (History)
2 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description sf 2016-02-05 11:19:51 UTC
For some days now emerge-delta-webrsync fails on all my computers:

# emerge-delta-webrsync

Looking for available base versions for a delta
Checking digest ...
fetching patches
Fetching file snapshot-20160203-20160204.patch.bz2 ...
Looking for available base versions for a delta
Checking digest ...
fetching patches
Fetching file snapshot-20160203-20160204.patch.bz2.md5sum ...
--2016-02-05 11:17:37--  http://distfiles.gentoo.org/snapshots/deltas/snapshot-20160203-20160204.patch.bz2.md5sum
Resolving distfiles.gentoo.org... 64.50.233.100, 137.226.34.46, 140.211.166.134, ...
Connecting to distfiles.gentoo.org|64.50.233.100|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 71 [application/x-bzip2]
Saving to: ‘/gentoo/distfiles/snapshot-20160203-20160204.patch.bz2.md5sum’

/gentoo/distfiles/snapshot 100%[=======================================>]      71  --.-KB/s   in 0s     

2016-02-05 11:17:38 (13.6 MB/s) - ‘/gentoo/distfiles/snapshot-20160203-20160204.patch.bz2.md5sum’ saved [71/71]

Fetching file snapshot-20160203-20160204.patch.bz2 ...
--2016-02-05 11:17:38--  http://distfiles.gentoo.org/snapshots/deltas/snapshot-20160203-20160204.patch.bz2
Resolving distfiles.gentoo.org... 64.50.233.100, 137.226.34.46, 140.211.166.134, ...
Connecting to distfiles.gentoo.org|64.50.233.100|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 238216 (233K) [application/x-bzip2]
Saving to: ‘/gentoo/distfiles/snapshot-20160203-20160204.patch.bz2’

/gentoo/distfiles/snapshot 100%[=======================================>] 232.63K   520KB/s   in 0.4s   

2016-02-05 11:17:38 (520 KB/s) - ‘/gentoo/distfiles/snapshot-20160203-20160204.patch.bz2’ saved [238216/238216]

Checking digest ...
Fetching file snapshot-20160204-20160205.patch.bz2 ...
--2016-02-05 11:17:38--  http://distfiles.gentoo.org/snapshots/deltas/snapshot-20160204-20160205.patch.bz2
Resolving distfiles.gentoo.org... 64.50.233.100, 137.226.34.46, 140.211.166.134, ...
Connecting to distfiles.gentoo.org|64.50.233.100|:80... connected.
HTTP request sent, awaiting response... 404 Not Found
2016-02-05 11:17:38 ERROR 404: Not Found.

failed fetching snapshot-20160204-20160205.patch.bz2
Fetching file portage-20160204.tar.bz2.md5sum ...
--2016-02-05 11:17:38--  http://distfiles.gentoo.org/snapshots/portage-20160204.tar.bz2.md5sum
Resolving distfiles.gentoo.org... 64.50.233.100, 137.226.34.46, 140.211.166.134, ...
Connecting to distfiles.gentoo.org|64.50.233.100|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 59 [application/x-bzip2]
Saving to: ‘/gentoo/distfiles/portage-20160204.tar.bz2.md5sum’

/gentoo/distfiles/portage- 100%[=======================================>]      59  --.-KB/s   in 0s     

2016-02-05 11:17:39 (12.4 MB/s) - ‘/gentoo/distfiles/portage-20160204.tar.bz2.md5sum’ saved [59/59]

Fetching file portage-20160204.tar.bz2.umd5sum ...
--2016-02-05 11:17:39--  http://distfiles.gentoo.org/snapshots/portage-20160204.tar.bz2.umd5sum
Resolving distfiles.gentoo.org... 64.50.233.100, 137.226.34.46, 140.211.166.134, ...
Connecting to distfiles.gentoo.org|64.50.233.100|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 55 [application/x-bzip2]
Saving to: ‘/gentoo/distfiles/portage-20160204.tar.bz2.umd5sum’

/gentoo/distfiles/portage- 100%[=======================================>]      55  --.-KB/s   in 0s     

2016-02-05 11:17:39 (11.6 MB/s) - ‘/gentoo/distfiles/portage-20160204.tar.bz2.umd5sum’ saved [55/55]

Fetching file portage-20160204.tar.bz2.gpgsig ...
--2016-02-05 11:17:39--  http://distfiles.gentoo.org/snapshots/portage-20160204.tar.bz2.gpgsig
Resolving distfiles.gentoo.org... 64.50.233.100, 137.226.34.46, 140.211.166.134, ...
Connecting to distfiles.gentoo.org|64.50.233.100|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 819 [application/x-bzip2]
Saving to: ‘/gentoo/distfiles/portage-20160204.tar.bz2.gpgsig’

/gentoo/distfiles/portage- 100%[=======================================>]     819  --.-KB/s   in 0s     

2016-02-05 11:17:39 (164 MB/s) - ‘/gentoo/distfiles/portage-20160204.tar.bz2.gpgsig’ saved [819/819]

verbosity level(1)
patch_type=8
disabling bufferless, patch_count(1) == 1 || forced_reorder(1)
size1=560670720, size2=560773120
reconstruction return=0, commands=98249
result was 98249 commands
versions size is 560773120
applied 1 patches
reordering commands? 1
reconstructing target file based off of dcbuff commands...
collapsing
processing src 0: 55174 commands.
processing src 1: 43075 commands.
reconstruction completed successfully
verifying uncompressed md5
Checking digest ...
recompressing ...
  /tmp/portage64/portage/delta-webrsync-8MDS2c/portage-20160204.tar:  7.437:1,  1.076 bits/byte, 86.55% saved, 560773120 in, 75398994 out.
Checking signature ...
gpg: WARNING: unsafe permissions on homedir `/etc/portage/gnupg'
gpg: Signature made Fri 05 Feb 2016 12:56:56 AM UTC using RSA key ID C9189250
gpg: BAD signature from "Gentoo Portage Snapshot Signing Key (Automated Signing Key)" [unknown]

As can be seen the uncompressed md5 is ok, but not the (re-)compressed md5:

# md5sum -c  /gentoo/distfiles/portage-20160204.tar.bz2.md5sum 
portage-20160204.tar.bz2: FAILED
md5sum: WARNING: 1 computed checksum did NOT match

As emerge-delta-webrsync and bzip2 have not been changed in portage for quite some time I would guess that snapshot compression on the gentoo server has changed (maybe parallel bzip2?).
Comment 1 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2016-02-05 11:35:50 UTC
Try: gpg --refresh-keys
Comment 2 sf 2016-02-05 12:35:54 UTC
This is not bug 570734. My keys are up to date:

# gpg --homedir /etc/portage/gnupg --list-keys
/etc/portage/gnupg/pubring.gpg
------------------------------
pub   4096R/96D8BF6D 2011-11-25 [expires: 2016-07-01]
uid       [ unknown] Gentoo Portage Snapshot Signing Key (Automated Signing Key)
sub   4096R/C9189250 2011-11-25 [expires: 2016-07-01]
Comment 3 ncl 2016-02-06 18:36:13 UTC
This is not a key issue; the tarball emerge-delta-webrsync produces is different from the original.

md5sums:
2852f5b070e5db382c22bdef98350ffa  /var/tmp/portage/delta-webrsync-F3zQ9V/portage-20160205.tar.bz2
2d54187ce61a7599f7a0017676b5f36a  portage-20160205.tar.bz2

After bunzip2:
0d4e56b12f1b9cfae871d7a5b59c5a39  portage-20160205-diff.tar
0d4e56b12f1b9cfae871d7a5b59c5a39  portage-20160205-orig.tar

sha256sums just in case:
29d3f073e6bc1a1dd2c5f7d453dedab5c9183b8043a055ce438a4e0d3ad70916  portage-20160205-diff.tar
29d3f073e6bc1a1dd2c5f7d453dedab5c9183b8043a055ce438a4e0d3ad70916  portage-20160205-orig.tar

bzip2 -vk9 (as emerge-delta-webrsync uses) yeilds the same file (expected):
2852f5b070e5db382c22bdef98350ffa  portage-20160205-diff.tar.bz2

lbzip2 -k9 yeilds:
2d54187ce61a7599f7a0017676b5f36a  portage-20160205-diff.tar.bz2

Which matches the bz2 in the repos.

So, it appears emerge-delta-webrsync users now needs to start using lbzip2.
Comment 4 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2016-02-06 18:42:58 UTC
Relying on specific output from recompression is a bad idea. I think we should start providing a GPG signature of the uncompressed tarball, and emerge-delta-webrsync should verify that instead.
Comment 5 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2016-02-06 21:44:06 UTC
I've updated the scripts now, so hopefully the next snapshot will work again. Please let me know if that's the case but don't close the bug since we really need to update the way e-d-w verifies tarballs.
Comment 6 sf 2016-02-09 10:15:26 UTC
emerge-delta-webrsync could successfully create portage-20160208.tar.bz2 from portage-20160207.tar.bz2.

Thanks for your effort, Michał.
Comment 7 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2023-08-19 15:55:39 UTC
(In reply to Michał Górny from comment #5)
> I've updated the scripts now, so hopefully the next snapshot will work
> again. Please let me know if that's the case but don't close the bug since
> we really need to update the way e-d-w verifies tarballs.

Heh, Zac mentioned this a while ago in bug 286373 too.