Bug 410935 – Portage's use of assert_sigpipe_ok in 'unpack' misses corrupted tarballs

• Status: RESOLVED DUPLICATE of bug 410861
• Product: Portage Development

[Description Ciaran McCreesh 2012-04-05 19:55:18 UTC]

See for example bug 410861. The tar file in question has trailing garbage, which you can see by passing "--ignore-zeros" to tar. The 'fix' for bug 309001 means Portage ignores this corruption. The assert_sigpipe_ok stuff needs to be reverted and replaced by full error checking, and any affected tarballs need to be replaced.

[Comment 1 Ulrich Müller 2012-04-08 20:55:27 UTC]

When executing the commands separately:
   $ xz -d foo.tar.xz
   $ tar xf foo.tar
neither of the two commands throws an error. So according to xz, it's a valid xz file. And according to tar, the uncompressed file is a valid archive.

I don't have a strong opinion on the matter, but maybe tar should be considered the authoritative entity that decides if an archive is valid?

[Comment 2 Ciaran McCreesh 2012-04-09 09:30:05 UTC]

You need to use "tar --ignore-zeros" to get a visible notification of the corruption.

[Comment 3 David Leverton 2012-04-09 14:09:46 UTC]

The tar documentation states

"When reading an archive, a reasonable system should properly handle an archive whose last record is shorter than the rest, or which contains garbage records after a zero block."

so I think we ought to accept these, annoying as it is.  Also, when tar itself runs the decompressor it ignores a SIGPIPE exit status, so for consistency we should do the same.

[Comment 4 Zac Medico 2012-04-21 21:24:03 UTC]

*** This bug has been marked as a duplicate of bug 410861 ***

[Comment 5 Ciaran McCreesh 2012-04-21 21:27:33 UTC]

You still need to do something about this... You're silently ignoring corruption in tarballs.

[Comment 6 Zac Medico 2012-04-21 21:29:05 UTC]

What can we do about it?

[Comment 7 Ciaran McCreesh 2012-04-21 21:33:08 UTC]

Use pipes for every compression format, reject every tarball that gives a SIGPIPE, and make ebuild maintainers fix the breakage. There are a whole bunch of corrupted tarballs on distfiles.g.o where ebuilds are ending up with partial uncompressions or similar, and developer testing obviously isn't good enough that it's been noticed.

[Comment 8 Zac Medico 2012-04-21 21:41:10 UTC]

In order to go through with this, we'll need some real data, not just a vague reference to "a whole bunch of corrupted tarballs."

[Comment 9 Zac Medico 2012-04-21 21:43:07 UTC]

How about if we trigger a "QA Notice" message when we detect the SIGPIPE? That would allow us to collect some real data.

[Comment 10 Ciaran McCreesh 2012-04-21 21:49:47 UTC]

http://paste.pocoo.org/show/579355/ was a run over a distfiles mirror, giving you a rough idea of the extent of the problem.