Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 760905 - unpacker.eclass: add zst support
Summary: unpacker.eclass: add zst support
Status: RESOLVED FIXED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Eclasses (show other bugs)
Hardware: All Linux
: Normal normal (vote)
Assignee: Gentoo's Team for Core System packages
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2020-12-20 17:16 UTC by Alexey Mishustin
Modified: 2020-12-21 05:15 UTC (History)
0 users

See Also:
Package list:
Runtime testing required: ---


Attachments
patch (0001-add-zst-support.patch,2.13 KB, patch)
2020-12-20 17:16 UTC, Alexey Mishustin
Details | Diff
patch v.2 (0001-unpacker.eclass-add-zst-support.patch,1.17 KB, patch)
2020-12-20 20:10 UTC, Alexey Mishustin
Details | Diff
patch v.2 - next (0002-unpacker.eclass-a-standard-comp-assigning.patch,821 bytes, patch)
2020-12-20 21:34 UTC, Alexey Mishustin
Details | Diff
patch v.3 (both v.2 commits squashed) (0001-unpacker.eclass-add-zst-support.patch,1.12 KB, patch)
2020-12-20 23:10 UTC, Alexey Mishustin
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Alexey Mishustin 2020-12-20 17:16:14 UTC
Created attachment 678951 [details, diff]
patch

Hi!

Please add zst [1] support to unpack.eclass.

I am going also to e-mail this patch to gentoo-dev and to create a PR on GitHub.

[1] - https://en.wikipedia.org/wiki/Zstandard
Comment 1 Alexey Mishustin 2020-12-20 17:18:24 UTC
(In reply to Alexey Mishustin from comment #0)

> Please add zst [1] support to unpack.eclass.

unpack -> unpacker
Comment 2 Mike Gilbert gentoo-dev 2020-12-20 17:42:06 UTC
Comment on attachment 678951 [details, diff]
patch

>From 53c8b275cab3b1213268bde10c9705dabd4c0e63 Mon Sep 17 00:00:00 2001
>From: halcon74 <halcon@tuta.io>
>Date: Sun, 20 Dec 2020 19:36:57 +0300
>Subject: [PATCH] add zst support

Please make the subject "unpacker.eclass: add zst support".

Also, please review GLEP 76 and add your Signed-off-by to the commit message.

https://www.gentoo.org/glep/glep-0076.html

>@@ -327,6 +327,21 @@ unpack_zip() {
> 	[[ $? -le 1 ]] || die "unpacking ${zip} failed (arch=unpack_zip)"
> }
> 
>+# @FUNCTION: unpack_zst
>+# @USAGE: <zst file>
>+# @DESCRIPTION:
>+# Unpack zst archives.
>+unpack_zst() {
>+	[[ $# -eq 1 ]] || die "Usage: ${FUNCNAME} <file>"
>+
>+	local zst=$(find_unpackable_file "$1")
>+	unpack_banner "${zst}"
>+	zstd -df "${zst}" -o "${P}.tar"
>+	tar -xpf "${P}.tar" -C "${WORKDIR}"
>+
>+	[[ $? -le 1 ]] || die "unpacking ${zst} failed (arch=unpack_zst)"
>+}

Please drop the unpack_zst function. It does not do anything special and is unnecessary.

> # @FUNCTION: _unpacker
> # @USAGE: <one archive to unpack>
> # @INTERNAL
>@@ -356,6 +371,8 @@ _unpacker() {
> 	*.lz)
> 		: ${UNPACKER_LZIP:=$(type -P plzip || type -P pdlzip || type -P lzip)}
> 		comp="${UNPACKER_LZIP} -dc" ;;
>+	*.zst)
>+		comp="zstd -df" ;;
> 	esac
> 
> 	# then figure out if there are any archiving aspects
>@@ -383,6 +400,8 @@ _unpacker() {
> 		;;
> 	*.zip)
> 		arch="unpack_zip" ;;
>+	*.zst)
>+		arch="unpack_zst" ;;

Please drop this. zstd is a compression format, not an archive format. Tarballs will be handled automatically.
Comment 3 Alexey Mishustin 2020-12-20 19:11:42 UTC
(In reply to Mike Gilbert from comment #2)

> Please drop the unpack_zst function. It does not do anything special and is
> unnecessary.

> Please drop this. zstd is a compression format, not an archive format.
> Tarballs will be handled automatically.

Interesting!

I am ready to remove the indicated changes.

But, unfortunately, I don't understand how is it supposed to use the patched eclass in this case? (Sorry that I have to ask about it)

If I:

1. remove these changes from the patched eclass
2. remove the call of unpack_zst "${A}" from src_unpack() from the ebuild with which I test the patched eclass

then I get the error

>>> Emerging (1 of 1) app-misc/worker-4.6.1::localrepo
 * worker-4.6.1.tar.zst BLAKE2B SHA512 size ;-) ...                                                                         [ ok ]
>>> Unpacking source...
>>> Unpacking worker-4.6.1.tar.zst to /var/tmp/portage/app-misc/worker-4.6.1/work
zstd: /var/tmp/portage/app-misc/worker-4.6.1/distdir/worker-4.6.1.tar: Permission denied
tar: This does not look like a tar archive
tar: Exiting with failure status due to previous errors
 * ERROR: app-misc/worker-4.6.1::localrepo failed (unpack phase):
 *   unpacking /var/tmp/portage/app-misc/worker-4.6.1/distdir/worker-4.6.1.tar.zst failed (comp=zstd -df arch=tar --no-same-owner -xof)
 * 
 * Call stack:
 *               ebuild.sh, line  125:  Called src_unpack
 *             environment, line 2146:  Called unpacker_src_unpack
 *             environment, line 3050:  Called unpacker
 *             environment, line 3045:  Called _unpacker 'worker-4.6.1.tar.zst'
 *             environment, line  428:  Called assert 'unpacking /var/tmp/portage/app-misc/worker-4.6.1/distdir/worker-4.6.1.tar.zst failed (comp=zstd -df arch=tar --no-same-owner -xof)'
 *   isolated-functions.sh, line   18:  Called die
 * The specific snippet of code:
 *              [[ $x -eq 0 ]] || die "$@"
Comment 4 Alexey Mishustin 2020-12-20 19:53:34 UTC
Ah! I figured it out :)
Comment 5 Alexey Mishustin 2020-12-20 20:10:40 UTC
Created attachment 678975 [details, diff]
patch v.2

A new version of the patch: 
- with added "Signed-off-by"
- without function unpack_zst
- without arch.
Comment 6 Alexey Mishustin 2020-12-20 21:34:04 UTC
Created attachment 678981 [details, diff]
patch v.2 - next

Thanks to junghans from GitHub, a patch of the next commit (consequent after patch v.2)
Comment 7 Mike Gilbert gentoo-dev 2020-12-20 22:19:59 UTC
Please combine the 2 commits into one. You can do so using an interactive rebase.

https://git-scm.com/book/en/v2/Git-Tools-Rewriting-History#_squashing
Comment 8 Alexey Mishustin 2020-12-20 23:10:22 UTC
Created attachment 678999 [details, diff]
patch v.3 (both v.2 commits squashed)

(In reply to Mike Gilbert from comment #7)
> Please combine the 2 commits into one. You can do so using an interactive
> rebase.
> 
> https://git-scm.com/book/en/v2/Git-Tools-Rewriting-History#_squashing

Done.
Comment 9 Larry the Git Cow gentoo-dev 2020-12-21 05:15:15 UTC
The bug has been closed via the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=ea98bf9dd7f825d440a9b0cb330721d9f854d404

commit ea98bf9dd7f825d440a9b0cb330721d9f854d404
Author:     halcon74 <halcon@tuta.io>
AuthorDate: 2020-12-20 20:08:38 +0000
Commit:     Mike Gilbert <floppym@gentoo.org>
CommitDate: 2020-12-21 05:12:06 +0000

    unpacker.eclass: add zst support
    
    Closes: https://bugs.gentoo.org/760905
    Closes: https://github.com/gentoo/gentoo/pull/18738
    Signed-off-by: Alexey Mishustin <halcon@tuta.io>
    Signed-off-by: Mike Gilbert <floppym@gentoo.org>

 eclass/unpacker.eclass | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)