Summary: | sys-apps/portage: add binary package compression_probe support for app-arch/zstd | ||
---|---|---|---|
Product: | Portage Development | Reporter: | Zac Medico <zmedico> |
Component: | Binary packages support | Assignee: | Portage team <dev-portage> |
Status: | RESOLVED FIXED | ||
Severity: | normal | Keywords: | InVCS |
Priority: | Normal | ||
Version: | 2.2 | ||
Hardware: | All | ||
OS: | All | ||
See Also: | https://bugs.gentoo.org/show_bug.cgi?id=634980 | ||
Whiteboard: | |||
Package list: | Runtime testing required: | --- | |
Bug Depends on: | |||
Bug Blocks: | 611328 | ||
Attachments: |
Support-for-zstandard-decompression.patch
Support-for-zstandard-decompression-v2.patch Support-for-zstandard-decompression-v3.patch |
Description
Zac Medico
2017-02-15 19:20:16 UTC
Created attachment 464436 [details, diff] Support-for-zstandard-decompression.patch Magic number taken from: https://github.com/facebook/zstd/blob/dev/doc/zstd_compression_format.md Comment on attachment 464436 [details, diff] Support-for-zstandard-decompression.patch >From 211caa3ab8fce4c5f284d81a6673930a13907635 Mon Sep 17 00:00:00 2001 >From: =?UTF-8?q?Manuel=20R=C3=BCger?= <mrueg@gentoo.org> >Date: Mon, 20 Feb 2017 16:08:44 +0100 >Subject: [PATCH] Add support for zstandard decompression > >X-Gentoo-Bug: 609462 >X-Gentoo-Bug-Url: https://bugs.gentoo.org/show_bug.cgi?id=609462 > >diff --git a/pym/portage/util/compression_probe.py b/pym/portage/util/compression_probe.py >index 74f74b163..b8c1a02e0 100644 >--- a/pym/portage/util/compression_probe.py >+++ b/pym/portage/util/compression_probe.py >@@ -18,6 +18,7 @@ _decompressors = { > "lzip": "lzip -d", > "lzop": "lzop -d", > "xz": "xz -d", >+ "zst": "zstd -d", Any reason you're naming it 'zst' instead of 'zstd'? Created attachment 464442 [details, diff] Support-for-zstandard-decompression-v2.patch Renamed it to zstd and fixed the magic number. Added Zstandard/LZ4 skippable frames as in https://github.com/file/file/blob/master/magic/Magdir/compress#L223 missed one? @@ -41,6 +43,7 @@ def compression_probe(f): lzip lzop xz + zst Do we really need to support files that begin with a skippable frame? If so, are such files distinguishable from LZ4 files? The frames are documented here: https://github.com/facebook/zstd/blob/v1.1.3/doc/zstd_compression_format.md#skippable-frames Apparently the first 4 bytes of a skippable frame can range from 0x184D2A50 to 0x184D2A5F, so we'd have to adjust the regex to account for that. Created attachment 464550 [details, diff]
Support-for-zstandard-decompression-v3.patch
Okay, let's only support non-skippable frames first. I think it's easier to add it later than to deprecate.
Fixed the comment as well.
Thanks, this is in the master branch: https://gitweb.gentoo.org/proj/portage.git/commit/?id=de0a26200b05b67cbeb1b8449acde18381fbb64b Fixed in portage-2.3.5. |