Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!

Bug 249059

Summary: Support source packages using lzip compression in ebuilds
Product: Gentoo Hosted Projects Reporter: Jon Bramley <brammers>
Component: PMS/EAPIAssignee: PMS/EAPI <pms>
Status: CONFIRMED ---    
Severity: enhancement CC: alexxy, antonio, hoelbezier, mgorny, sam
Priority: High    
Version: unspecified   
Hardware: All   
OS: Linux   
URL: http://www.nongnu.org/lzip/lzip.html
See Also: https://bugs.gentoo.org/show_bug.cgi?id=501912
Whiteboard:
Package list:
Runtime testing required: ---
Bug Depends on:    
Bug Blocks: 174380    
Attachments: lzip-1.1.ebuild
Integrates app-arch/lzip into portage

Description Jon Bramley 2008-11-27 09:38:55 UTC
The latest version of pam_mount (1.4) is only being released in .tar.lz (lzip) format, which is not currently supported by portage. Can it be added?

Reproducible: Always

Steps to Reproduce:
1. Create an ebuild where the source has a .lz extension
2. Attempt to build the package
3.

Actual Results:  
It fails with an unknown extension error

Expected Results:  
It extracts the source using lzip and continues the build as normal
Comment 1 Zac Medico gentoo-dev 2008-11-27 09:58:42 UTC
It seems similar to lzma-utils, but incompatible.
Comment 2 Gilles Dartiguelongue (RETIRED) gentoo-dev 2008-12-09 10:58:22 UTC
Created attachment 174704 [details]
lzip-1.1.ebuild

quick ebuild. lzip generates archives that are incompatible with lzma because it integrates checksumming (at least that's what I was told).
Comment 3 SpanKY gentoo-dev 2009-02-02 19:46:18 UTC
ive added the lzip package, but i dont think there's much point in adding it to the EAPI considering it'll be chucked as soon as xz comes out (which will in turn chuck lzma)

http://www.mail-archive.com/automake@gnu.org/msg14399.html
Comment 4 Daniel J. 2009-03-24 19:11:22 UTC
Created attachment 186141 [details, diff]
Integrates app-arch/lzip into portage

lzip is a different app than lzma.  I'd like to submit an ebuild for GNU Moe, for example, but the only tarball format is now .tar.lz, which lzma will NOT decompress.  This results in a very ugly src_unpack() in the ebuild; making portage more flexible seems much more elegant.

I've attached a minimal patch for sys-apps/portage enabling lzip when a tarball ends in .lz.
Comment 5 Daniel J. 2009-03-24 19:12:49 UTC
Or perhaps I misunderstood.  When xz comes out, will it handle .lz files?
Comment 6 Antonio Diaz Diaz 2009-10-06 11:20:48 UTC
xz is not at all related to lzip and most probably will never handle .lz files. In fact lzip seems to be the right LZMA tool to support in ebuilds. See http://lpar.ath0.com/2009/09/25/documentation-as-an-indicator-of-code-quality/
Comment 7 Antonio Diaz Diaz 2010-09-14 17:46:00 UTC
FYI, lzip has been marked stable for x86 and amd64.
https://bugs.gentoo.org/show_bug.cgi?id=329319
Comment 8 Ulrich Müller gentoo-dev 2010-10-14 08:32:22 UTC
(In reply to comment #0)
> The latest version of pam_mount (1.4) is only being released in .tar.lz
> (lzip) format, which is not currently supported by portage. Can it be added?

Meanwhile pam_mount has switched to .tar.xz format.

There's only one ebuild in the tree (app-text/ocrad-0.18) that uses an .lz distfile. Should we really add support for a format that apparently is dying?
Comment 9 Christian Faulhammer (RETIRED) gentoo-dev 2010-10-14 09:42:08 UTC
(In reply to comment #8)
> There's only one ebuild in the tree (app-text/ocrad-0.18) that uses an .lz
> distfile. Should we really add support for a format that apparently is dying?

 Definitely not.
Comment 10 Antonio Diaz Diaz 2010-10-15 16:09:53 UTC
(In reply to <a href="http://bugs.gentoo.org/show_bug.cgi?id=249059#c8">comment #8</a>)
> Should we really add support for a format that apparently is dying?

Sorry but this is the most superficial analysis I have ever read. Lzip is a relatively recent format that is not yet widely used because it is not yet widely supported and vice versa.

Lzip is a high quality family of programs built around a common format that offers, among other things, recovery features beyond those of any other compressor.

So the question is, are you going to decide about supporting a format based on the format's virtues and defects, or are you going to decide based on popularity and contribute to make free software <a href="http://www.guardian.co.uk/technology/2008/sep/29/cloud.computing.richard.stallman">more fashion-driven than women's fashion</a>.
Comment 11 Ciaran McCreesh 2010-10-15 16:18:31 UTC
We make the decision about what to support in EAPIs based upon whether or not enough ebuilds will use the feature to make it worth the maintenance cost.

If we add EAPI support for lzip, we have to be sure that five years from now, we'll not have to worry about finding an lzip program that is still actively maintained.

In the mean time, ebuilds can use manual uncompression for lzip. Once a lot of ebuilds start doing that, *then* we can move it into an EAPI.
Comment 12 Antonio Diaz Diaz 2010-10-15 17:20:39 UTC
Thanks for the aclaration.

I guarantee you that in five years from now lzip will be still actively maintained.

In fact, given the history of the various LZMA compressors for GNU/Linux, it may well happen that lzip is the only LZMA compressor that is still actively maintained in five years from now.
Comment 13 Ciaran McCreesh 2010-10-15 17:27:04 UTC
Then one year from now, when lzip is being used by a good number of packages, it will be time to evaluate whether it's time to move support out of ebuilds and into EAPI.
Comment 14 Ulrich Müller gentoo-dev 2010-10-15 17:48:21 UTC
(In reply to comment #10)
> Sorry but this is the most superficial analysis I have ever read. Lzip is a
> relatively recent format that is not yet widely used because it is not yet
> widely supported and vice versa.
> 
> Lzip is a high quality family of programs built around a common format that
> offers, among other things, recovery features beyond those of any other
> compressor.

Please don't misunderstand my statement. This is in no way meant as a judgement about quality of lzip vs. xz-utils.

However, this bug was opened almost two years ago, even before the first version of xz-utils was released. Meanwhile, xz has gained enough popularity that more than 300 ebuilds in the portage tree are using it. Whereas lzip is used by one single ebuild only, and even the package that was the original case for this bug has switched from lzip to xz.

So as Ciaran said, at the moment the additional maintenance burden would not be justified. (And I don't see a trend that this would change anytime soon.)
Comment 15 Ulrich Müller gentoo-dev 2011-10-21 07:09:04 UTC
(In reply to comment #13)
> Then one year from now, when lzip is being used by a good number of packages,
> it will be time to evaluate whether it's time to move support out of ebuilds
> and into EAPI.

One more year has passed, and lzip is still being used by a single package only (app-text/ocrad, which happens to be maintained by lzip's author).

Closing as WONTFIX. Feel free to reopen when this situation has changed.
Comment 16 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2021-10-31 12:18:30 UTC
*** Bug 821076 has been marked as a duplicate of this bug. ***
Comment 17 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2021-10-31 12:24:48 UTC
10 years later, I can count 6 packages using .lz in ::gentoo.  icewm is the only one not maintained by lzip's author.
Comment 18 Ulrich Müller gentoo-dev 2021-10-31 14:52:23 UTC
The last format for which we added unpack() support was *.xz in EAPI 3. It was used by about 4000 distfiles back then (and is used by more than 13000 today).

OTOH, we've removed 7zip support in EAPI 8. I count 60 *.7z distfiles, which is an order of magnitude more than *.lz.

At least for now, unpacker.eclass looks like a better place to support such a little used format.
Comment 19 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2021-10-31 20:06:53 UTC
For the record, the only reason we've kept .lzma is because xz-utils supports it.  As a curiosity, a few years ago I've made patches to support .lz in xz-utils but it went nowhere.  Basically, it took a few years for upstream to review it, and when they did, I didn't have time to work on it.  If you want to take the effort over, I can try digging them up.
Comment 20 Ulrich Müller gentoo-dev 2022-12-14 09:48:01 UTC
Now that >=xz-utils-5.4.0 supports unpacking of *.lz files, maybe we should reconsider? It's basically the same situation as for *.lzma now, which we also keep because supporting it has zero cost.
Comment 21 Larry the Git Cow gentoo-dev 2022-12-14 10:18:13 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=7713a9f0fc204a09d6b2a57a46e520f07bed72be

commit 7713a9f0fc204a09d6b2a57a46e520f07bed72be
Author:     Sam James <sam@gentoo.org>
AuthorDate: 2022-12-13 21:02:58 +0000
Commit:     Michał Górny <mgorny@gentoo.org>
CommitDate: 2022-12-14 10:16:17 +0000

    unpacker.eclass: support >=app-arch/xz-utils-5.4.0 for lzip decompression
    
    >=app-arch/xz-utils-5.4.0 supports lzip decompression (not compression).
    
    Add support for unpacker.eclass to handle it for .lz files.
    
    Note that xz-utils is part of @system (and PMS requires that .xz is unpackable),
    while most users do not have lzip and friends installed.
    
    (Note that xz does not (currently, but does not plan on either) implement
    parallel decompression for .lz, but most .lz distfiles are small, so this
    isn't an issue.)
    
    Historically, we've often repacked .lz distfiles for important packages
    to avoid users needing to install app-arch/lzip for a single distfile,
    so this avoids the need for that (although I've not done it out of
    principle for things like sys-apps/ed).
    
    Bug: https://bugs.gentoo.org/249059
    Bug: https://bugs.gentoo.org/485462
    Bug: https://bugs.gentoo.org/501912
    Bug: https://bugs.gentoo.org/502990
    Bug: https://bugs.gentoo.org/545344
    Signed-off-by: Sam James <sam@gentoo.org>
    Signed-off-by: Michał Górny <mgorny@gentoo.org>

 eclass/unpacker.eclass | 31 +++++++++++++++++++++++++++----
 1 file changed, 27 insertions(+), 4 deletions(-)