Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 833121 - Support source packages using zstd compression in ebuilds
Summary: Support source packages using zstd compression in ebuilds
Status: RESOLVED WONTFIX
Alias: None
Product: Gentoo Hosted Projects
Classification: Unclassified
Component: PMS/EAPI (show other bugs)
Hardware: All Linux
: Normal enhancement
Assignee: Package Manager Specification
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2022-02-11 10:51 UTC by giskard
Modified: 2023-04-07 02:13 UTC (History)
4 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description giskard 2022-02-11 10:51:33 UTC
as zstd is much more fast than gzip and many others, and provide even better compress ratio than gzip and many others at the same time, I think it's time to integrate it to the core system set, and prefer it over the others

Reproducible: Always
Comment 1 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2022-02-11 11:10:40 UTC
It's not clear to me what you mean by default compression method. We use tarballs and distribution methods provided by upstreams.

I think Portage already changed its default for new installs for generated binary packages to use zstd.
Comment 2 Mike Gilbert gentoo-dev 2022-02-11 16:48:22 UTC
Yeah, it's very unclear what you are proposing.
Comment 3 giskard 2022-02-12 18:49:17 UTC
there I find a font and want to write an ebuild for it, the upstream provide zstd tarball, but when I use the zstd tarball, it failed during unpack stage with unknown file format issue, I have to giveup and use tar.gz instead. Here I find the related wiki:
 https://devmanual.gentoo.org/ebuild-writing/functions/src_unpack/index.html

zst is not listed there, looks like `unpack` is implemented in somewhere of the core system set, it does not recognize zstd compressed tarball at the time.

And currently zstd seems not in the default system package set, as a result, it can't be the DEFAULT compression method for the build system, or you have to list zstd in BDEPEND manually.

Well, as gzip, bzip and etc. works out of the box, the better zstd deserves the same treatment.
Comment 4 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2022-02-12 19:09:04 UTC
(In reply to giskard from comment #3)
> there I find a font and want to write an ebuild for it, the upstream provide
> zstd tarball, but when I use the zstd tarball, it failed during unpack stage
> with unknown file format issue, I have to giveup and use tar.gz instead.
> Here I find the related wiki:
>  https://devmanual.gentoo.org/ebuild-writing/functions/src_unpack/index.html
> 
> zst is not listed there, looks like `unpack` is implemented in somewhere of
> the core system set, it does not recognize zstd compressed tarball at the
> time.

I understand this part and it's a reasonable request. But note that you could use unpacker.eclass for this.

> 
> And currently zstd seems not in the default system package set, as a result,
> it can't be the DEFAULT compression method for the build system, or you have
> to list zstd in BDEPEND manually.
> 
> Well, as gzip, bzip and etc. works out of the box, the better zstd deserves
> the same treatment.

This part doesn't make sense. "Default compression method".. for what?

See what I said about tarballs. Please file a new bug for specific other requests.
Comment 5 giskard 2022-02-14 09:37:09 UTC
(In reply to Sam James from comment #4)
> (In reply to giskard from comment #3)
> > there I find a font and want to write an ebuild for it, the upstream provide
> > zstd tarball, but when I use the zstd tarball, it failed during unpack stage
> > with unknown file format issue, I have to giveup and use tar.gz instead.
> > Here I find the related wiki:
> >  https://devmanual.gentoo.org/ebuild-writing/functions/src_unpack/index.html
> > 
> > zst is not listed there, looks like `unpack` is implemented in somewhere of
> > the core system set, it does not recognize zstd compressed tarball at the
> > time.
> 
> I understand this part and it's a reasonable request. But note that you
> could use unpacker.eclass for this.

I thought unpacker is just the default unpack. great! it works for .tar.zst, though .tzst is not recognized. 

> 
> > 
> > And currently zstd seems not in the default system package set, as a result,
> > it can't be the DEFAULT compression method for the build system, or you have
> > to list zstd in BDEPEND manually.
> > 
> > Well, as gzip, bzip and etc. works out of the box, the better zstd deserves
> > the same treatment.
> 
> This part doesn't make sense. "Default compression method".. for what?

maybe call it "the preferred" is better, if include it into system set and add the support in original src_unpack, it can be regarded as the "supported out of the box".

as for the "for what?", of course for the far more efficient than general gzip

> 
> See what I said about tarballs. Please file a new bug for specific other
> requests.
Comment 6 giskard 2022-12-07 04:28:39 UTC
not actually a problem, close
Comment 7 Luigi Semenzato 2023-04-06 20:33:47 UTC
Can we reopen this bug?  Just because one can use "unpacker()" from unpacker.eclass, it doesn't seem a good reason to fail when the default unpack is used.  The zstd format is an improvement, and is popular.

Is it because the additional dependency on the zstd package is undesirable?

Thanks!
Comment 8 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2023-04-07 02:13:11 UTC
(In reply to Luigi Semenzato from comment #7)
> Can we reopen this bug?  Just because one can use "unpacker()" from
> unpacker.eclass, it doesn't seem a good reason to fail when the default
> unpack is used.  The zstd format is an improvement, and is popular.
> 
> Is it because the additional dependency on the zstd package is undesirable?

The issue is that not many pieces of software are distributed (at least exclusively, but really at all) with .zst, so making the package manager unconditionally depend on it is seen as controversial.

That said, Portage already depends on it by default for its default of BINPKG_COMPRESS="zstd" (it probably should've bene xz given it already depends on it, but that ship has sailed, i think.)