* Running $myvarcmd >>> Syncing repository 'gentoo' into '/usr/portage'... Fetching most recent snapshot ... Trying to retrieve 20160201 snapshot from http://distfiles.gentoo.org ... Fetching file portage-20160201.tar.xz.md5sum ... Fetching file portage-20160201.tar.xz.gpgsig ... Fetching file portage-20160201.tar.xz ... Checking digest ... Checking signature ... gpg: Signature made Tue Feb 2 01:49:45 2016 CET using RSA key ID C9189250 gpg: Good signature from "Gentoo Portage Snapshot Signing Key (Automated Signing Key)" [unknown] gpg: WARNING: This key is not certified with a trusted signature! gpg: There is no indication that the signature belongs to the owner. Primary key fingerprint: DCD0 5B71 EAB9 4199 527F 44AC DB6B 8C1F 96D8 BF6D Subkey fingerprint: E1D6 ABB6 3BFC FB4B A02F DF1C EC59 0EEA C918 9250 Getting snapshot timestamp ... Syncing local tree ... emerge-webrsync: error: tar failed to extract the image. tarball is corrupt? (/var/tmp/portage/webrsync-eBIOwt/portage-20160201.tar.xz) Trying to retrieve 20160131 snapshot from http://distfiles.gentoo.org ... Fetching file portage-20160131.tar.xz.md5sum ... Fetching file portage-20160131.tar.bz2.md5sum ... Fetching file portage-20160131.tar.bz2.gpgsig ... Fetching file portage-20160131.tar.bz2 ... Checking digest ... Checking signature ... gpg: Signature made Mon Feb 1 01:48:54 2016 CET using RSA key ID C9189250 gpg: Good signature from "Gentoo Portage Snapshot Signing Key (Automated Signing Key)" [unknown] gpg: WARNING: This key is not certified with a trusted signature! gpg: There is no indication that the signature belongs to the owner. Primary key fingerprint: DCD0 5B71 EAB9 4199 527F 44AC DB6B 8C1F 96D8 BF6D Subkey fingerprint: E1D6 ABB6 3BFC FB4B A02F DF1C EC59 0EEA C918 9250 Getting snapshot timestamp ... Syncing local tree ... BTW: Why does it switch to the older portage-20160131.tar.bz2 if the newer portage-20160201.tar.bz2 is available?
*** Bug 573678 has been marked as a duplicate of this bug. ***
Please paste the output of: tar --version xz --version
I had the same thing, tar (GNU tar) 1.27.1 xz (XZ Utils) 5.0.8 liblzma 5.0.8
testing this app-arch/tar-1.28-r1::gentoo app-arch/xz-utils-5.2.2::gentoo still get the error Syncing local tree ... emerge-webrsync: error: tar failed to extract the image. tarball is corrupt? (/var/tmp/portage/webrsync-6CvNwh/portage-20160201.tar.xz)
So this is a bug in emerge-webrsync script. I've been able to reproduce the issue doing: $ xz -cd portage-20160201.tar.xz | tar -t >/dev/null $ echo ${PIPESTATUS[*]} 141 0 Long story short, 'xz' is killed by SIGPIPE when tar reaches the end of the archive and emerge-webrsync wrongly assumes it failed. This most likely started happening because snapshots are now compressed using pixz.
I'm seeing this on one of my gentoo machines, 3 others synced just fine today, all 4 using webrsync-gpg, the same mirror and the same versions of sys-apps/portage (2.2.26), app-arch/tar (1.27.1-r2) and app-arch/xz-utils (5.0.8), all with the same USE-Flags on all 4 machines. Any ideas, where else to look for the difference?
Created attachment 424520 [details] Log of syncing with sync-type webrsync (In reply to Michał Górny from comment #5) > [...] > This most likely started happening because snapshots are now compressed > using pixz. Could this please be redone for the time being until stage3 tarballs have a version of emerge-webrsync which is compatible with pixz compressed snapshot tarballs? The point is that Gentoo handbook / Installation / Installing base system in section "Installing a Portage snapshot" ( https://wiki.gentoo.org/wiki/Handbook:AMD64/Installation/Base#Installing_a_Portage_snapshot) recommends emerge-webrsync for the first installation of a snapshot. Presently this would not install the latest tar.xz snapshot but only a three days old tar.bz2 snapshot (see my attachment).
What is the plan for fixing this problem? As I see it the old snapshot format that works with current emerge-webrsync should be restored. Emerge-webrsync can then be upgraded to a version that works with the new format. Then you wait for a year before rolling out the new format so users got a chance to upgrade to the new emerge-webrsync.
I'd be interested in a real fix or even a real workaround, but in the meantime here's a stupid workaround, attached. It makes emerge-webrsync pass over .xz compressed snapshots and prefer .bz2 instead.
Created attachment 424584 [details, diff] Make emerge-webrsync skip over .xz archives, which are currently broken.
Created attachment 424590 [details] webrsync with app-arch/tarsync installed still works (In reply to Hank Leininger from comment #10) > Created attachment 424584 [details, diff] [details, diff] > Make emerge-webrsync skip over .xz archives, which are currently broken. On a freshly expanded and chrooted-into stage3 this seems to be the only possibility by now. For an existing installation one could emerge app-arch/tarsync as described in <https://wiki.gentoo.org/wiki//etc/portage/repos.conf/webrsync.conf>. Just tested and it works for me.
Robin has just updated the script to use threaded 'xz' instead of 'pixz'. Hopefully this will fix the issue when the next snapshot comes out.
Thanks to Robin! Just tested without app-arch/tarsync and it works again using the tar.xz tarball: * Running $myvarcmd >>> Syncing repository 'gentoo' into '/usr/portage'... Fetching most recent snapshot ... Trying to retrieve 20160204 snapshot from http://ftp-stud.hs-esslingen.de/pub/Mirrors/gentoo ... Fetching file portage-20160204.tar.xz.md5sum ... Fetching file portage-20160204.tar.xz.gpgsig ... Fetching file portage-20160204.tar.xz ... Checking digest ... Checking signature ... gpg: Signature made Fri Feb 5 01:56:57 2016 CET using RSA key ID C9189250 gpg: Good signature from "Gentoo Portage Snapshot Signing Key (Automated Signing Key)" [unknown] gpg: WARNING: This key is not certified with a trusted signature! gpg: There is no indication that the signature belongs to the owner. Primary key fingerprint: DCD0 5B71 EAB9 4199 527F 44AC DB6B 8C1F 96D8 BF6D Subkey fingerprint: E1D6 ABB6 3BFC FB4B A02F DF1C EC59 0EEA C918 9250 Getting snapshot timestamp ... Syncing local tree ... Number of files: 208,115 (reg: 180,852, dir: 27,263) Number of created files: 83 (reg: 79, dir: 4) Number of deleted files: 67 (reg: 65, dir: 2) Number of regular files transferred: 39,557 Total file size: 409.37M bytes Total transferred file size: 85.71M bytes Literal data: 85.71M bytes Matched data: 0 bytes File list size: 3.10M File list generation time: 0.001 seconds File list transfer time: 0.000 seconds Total bytes sent: 51.27M Total bytes received: 786.67K sent 51.27M bytes received 786.67K bytes 127.43K bytes/sec total size is 409.37M speedup is 7.86 Cleaning up ... Performing Global Updates (Could take a couple of minutes if you have a lot of binary packages.) === Sync completed for gentoo
I can also confirm that the current snapshot ('portage-20160204.tar.xz') works fine here. Thanks!