Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 703460 - sys-apps/portage-2.3.82 switches to gentoo-YYYYMMDD.tar.xz snapshots, which are broken
Summary: sys-apps/portage-2.3.82 switches to gentoo-YYYYMMDD.tar.xz snapshots, which a...
Status: RESOLVED OBSOLETE
Alias: None
Product: Gentoo Infrastructure
Classification: Unclassified
Component: Git (show other bugs)
Hardware: All Linux
: Normal major (vote)
Assignee: Gentoo Infrastructure
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2019-12-21 07:27 UTC by Hank Leininger
Modified: 2021-08-26 00:03 UTC (History)
3 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Hank Leininger 2019-12-21 07:27:46 UTC
After upgrading to sys-apps/portage-2.3.82, a subsequent emerge --sync will result in a broken portage tree.

This is because a) 2.3.82 switches to using gentoo-YYYYMMDD.tar.xz snapshots instead of portage-YYYYMMDD.tar.xz snapshots (see https://bugs.gentoo.org/693454), and the latter snapshots are (currently?) broken - long filenames are truncated.

The gentoo- and portage- tarballs ought to be about identical, with the exception of different top-level directories.  But they are not:

$ wget https://gentoo.osuosl.org/snapshots/portage-20191220.tar.xz
$ wget https://gentoo.osuosl.org/snapshots/gentoo-20191220.tar.xz
# [Snip verifying their PGP fingerprints.]
$ diff -U0 <(tar tf portage-20191220.tar.xz | cut -d/ -f2- | sort) <(tar tf gentoo-20191220.tar.xz | cut -d/ -f2- | sort) | egrep -A10 -m4 ^@
@@ -815 +815 @@
-app-accessibility/at-spi2-core/files/at-spi2-core-2.0.2-disable-teamspaces-test.patch
+app-accessibility/at-spi2-core/files/at-spi2-core-2.0.2-disable-teamspaces-test.patc
@@ -1523 +1523 @@
-app-admin/gnome-abrt/files/0001-Remove-Expert-mode-and-the-remaining-Analyze-code.patch
+app-admin/gnome-abrt/files/0001-Remove-Expert-mode-and-the-remaining-Analyze-code.pa
@@ -2370 +2370 @@
-app-admin/system-config-printer/files/system-config-printer-1.5.12-check-for-null.patch
+app-admin/system-config-printer/files/system-config-printer-1.5.12-check-for-null.pa
@@ -2378,2 +2378,2 @@
-app-admin/system-tools-backends/files/system-tools-backends-2.8.2-cve-2008-4311.patch
-app-admin/system-tools-backends/files/system-tools-backends-2.8.2-default-permissions.patch
+app-admin/system-tools-backends/files/system-tools-backends-2.8.2-cve-2008-4311.patc
+app-admin/system-tools-backends/files/system-tools-backends-2.8.2-default-permission
@@ -2476 +2476 @@
-app-admin/webapp-config/files/webapp-config-1.53-sources-function.sh-from-lib-gentoo.patch
+app-admin/webapp-config/files/webapp-config-1.53-sources-function.sh-from-lib-gentoo
@@ -2992 +2992 @@
-app-arch/snappy/files/snappy-1.1.7-0001-cmake-Add-missing-linking-to-GTEST_LIBRARIES.patch
+app-arch/snappy/files/snappy-1.1.7-0001-cmake-Add-missing-linking-to-GTEST_LIBRARIES


So, it is the gentoo-YYYYMMDD.tar.xz snapshots that are broken, which probably counts as a releng issue, but this is only exposed when upgrading to the latest portage version.  Masking portage-2.3.82 and falling back to .81 will continue using the old portage-* snapshots, which still work fine.

N.B. I only looked at 20191219 and 20191220 snapshots, I don't know how far back this goes.


There is another oddity with the gentoo- snapshots; they contain a mix of top-level dirs, which does not appear to be intentional:

$ tar -tf portage-20191220.tar.xz | cut -d/ -f1 | sort | uniq -c
 160755 portage

vs

$ tar -tf gentoo-20191220.tar.xz | cut -d/ -f1 | sort | uniq -c
 160544 gentoo-20191220
    211 portage

This should have no user-visible impact since the gentoo-* tarballs are unpacked with --strip-components=1, so the different toplevels are discarded during unpacking.  But it probably indicates another problem with the releng process that builds them.
Comment 1 Robin Johnson archtester Gentoo Infrastructure gentoo-dev Security 2019-12-21 21:04:14 UTC
https://gitweb.gentoo.org/infra/mastermirror-scripts.git/commit/?id=32352ad5d45276c00ca05472d4bddda4b96c4c22 should fix it.

Just want some quick review of that commit plus the refactor:
https://gitweb.gentoo.org/infra/mastermirror-scripts.git/commit/?id=c571b6abdf2a73db93d765a1355ff02da7a0d5e8

Tagged as 20191221T130302Z, but not deployed via puppet pending review
Comment 2 Zac Medico gentoo-dev 2019-12-21 22:06:46 UTC
If we delete those broken snapshots from the mirrors, it will shield users from the breakage.
Comment 3 Robin Johnson archtester Gentoo Infrastructure gentoo-dev Security 2019-12-24 22:11:11 UTC
zmedico:
I have removed the following snapshots:
-rw-r--r--  1 gmirror gmirror 45480028 Dec 17 00:57 gentoo-20191216.tar.xz
-rw-r--r--  1 gmirror gmirror      963 Dec 17 00:57 gentoo-20191216.tar.xz.gpgsig
-rw-r--r--  1 gmirror gmirror       57 Dec 17 00:57 gentoo-20191216.tar.xz.md5sum
-rw-r--r--  1 gmirror gmirror       54 Dec 17 00:57 gentoo-20191216.tar.xz.umd5sum
-rw-r--r--  1 gmirror gmirror 45549596 Dec 18 00:57 gentoo-20191217.tar.xz
-rw-r--r--  1 gmirror gmirror      963 Dec 18 00:57 gentoo-20191217.tar.xz.gpgsig
-rw-r--r--  1 gmirror gmirror       57 Dec 18 00:57 gentoo-20191217.tar.xz.md5sum
-rw-r--r--  1 gmirror gmirror       54 Dec 18 00:57 gentoo-20191217.tar.xz.umd5sum
-rw-r--r--  1 gmirror gmirror 45593428 Dec 19 00:57 gentoo-20191218.tar.xz
-rw-r--r--  1 gmirror gmirror      963 Dec 19 00:57 gentoo-20191218.tar.xz.gpgsig
-rw-r--r--  1 gmirror gmirror       57 Dec 19 00:57 gentoo-20191218.tar.xz.md5sum
-rw-r--r--  1 gmirror gmirror       54 Dec 19 00:57 gentoo-20191218.tar.xz.umd5sum
-rw-r--r--  1 gmirror gmirror 45633820 Dec 20 00:57 gentoo-20191219.tar.xz
-rw-r--r--  1 gmirror gmirror      963 Dec 20 00:57 gentoo-20191219.tar.xz.gpgsig
-rw-r--r--  1 gmirror gmirror       57 Dec 20 00:57 gentoo-20191219.tar.xz.md5sum
-rw-r--r--  1 gmirror gmirror       54 Dec 20 00:57 gentoo-20191219.tar.xz.umd5sum
-rw-r--r--  1 gmirror gmirror 45625640 Dec 21 00:57 gentoo-20191220.tar.xz
-rw-r--r--  1 gmirror gmirror      963 Dec 21 00:57 gentoo-20191220.tar.xz.gpgsig
-rw-r--r--  1 gmirror gmirror       57 Dec 21 00:57 gentoo-20191220.tar.xz.md5sum
-rw-r--r--  1 gmirror gmirror       54 Dec 21 00:57 gentoo-20191220.tar.xz.umd5sum
Comment 4 Alec Warner archtester Gentoo Infrastructure gentoo-dev Security 2021-08-25 23:53:17 UTC
I think this is no longer a problem.
Comment 5 Hank Leininger 2021-08-26 00:03:03 UTC
(In reply to Alec Warner from comment #4)
> I think this is no longer a problem.

Agreed, I've never seen anything like this happen again after the switch away from Archive::Tar::Stream.