Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 335340 - portage should allow configuration of bzip2/bunzip2 commands for calling pbzip2
Summary: portage should allow configuration of bzip2/bunzip2 commands for calling pbzip2
Alias: None
Product: Portage Development
Classification: Unclassified
Component: Enhancement/Feature Requests (show other bugs)
Hardware: All All
: High normal (vote)
Assignee: Portage team
Keywords: InVCS
: 264258 (view as bug list)
Depends on:
Blocks: 335925
  Show dependency tree
Reported: 2010-08-31 00:17 UTC by Zac Medico
Modified: 2010-09-06 05:55 UTC (History)
3 users (show)

See Also:
Package list:
Runtime testing required: ---

Squashed and cleaned 2.1 backport (portage-bzip2.patch,8.39 KB, patch)
2010-09-02 17:48 UTC, Zdenek Behan
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Zac Medico gentoo-dev 2010-08-31 00:17:19 UTC
We need PORTAGE_BZIP2_COMMAND and PORTAGE_BUNZIP2_COMMAND variables where the user can set the command and arguments. Portage will append it's own arguments (bzip2/bunzip2 compatible) as necessary.
Comment 1 Zac Medico gentoo-dev 2010-08-31 00:35:11 UTC
PORTAGE_BUNZIP2_COMMAND is provided for flexibility, but if it's unset then we can fall back to $PORTAGE_BZIP2_COMMAND -d.
Comment 3 Zac Medico gentoo-dev 2010-08-31 05:05:27 UTC
Note: ecompress and ecompressdir don't have support for this because they already have separate PORTAGE_COMPRESS and PORTAGE_COMPRESS_FLAGS configuration variables.
Comment 4 Zdenek Behan 2010-08-31 22:54:29 UTC
I think it makes most sense for binary packages only anyway. And if it allows the override of unpacking the upstream tar.bz2 tarballs using the parallel uncompress, even better. Maybe two variables for GZIP would close the chain, i hear there's also a parallel implementation of gzip (pigz), so that people obsessed with parallelism can make use of the most of their computers.
Comment 5 Zac Medico gentoo-dev 2010-09-01 23:06:48 UTC
This is fixed in 2.2_rc73, but I'll leave this bug open until it's fixed in an unmasked version.
Comment 6 Zdenek Behan 2010-09-02 17:48:20 UTC
Created attachment 245771 [details, diff]
Squashed and cleaned 2.1 backport

I've ported/cleaned up a patch that works for me and seems to do the right thing in (the one chromeOS uses), which I'm experimentally epatching to our portage.
Comment 7 Zdenek Behan 2010-09-02 22:37:24 UTC
I ran into issues where portage, when upgrading cuts its own branch under itself.

While enviroment is preserved correctly in the python, right after install phase, when emerge preserves the enviroment into enviroment.bz2, this fails because the new is used, but old make.globals are still sourced.

This technically works, but breaks the enviroment in package database and produces a backtrace.

What we did is basically S/${PORTAGE_BZIP2_COMMAND}/${PORTAGE_BZIP2_COMMAND:-bzip2}, in all places. That means that in addition to make.globals setting a default, there's a fail-safe last resort default for all instances of the variable. I'm not sure if the same is a problem for the 2.2 portage patch.
Comment 8 Zac Medico gentoo-dev 2010-09-02 22:48:56 UTC
(In reply to comment #7)
> What we did is basically
> S/${PORTAGE_BZIP2_COMMAND}/${PORTAGE_BZIP2_COMMAND:-bzip2}, in all places. 

I already had something like that here:;a=commit;h=9437c8071af071ebd753c4ed29ce53948dcb9b1e

There's also related commits here:;a=commit;h=75dab994c4a3e7c38ea29cc633706c435991da05;a=commit;h=c050d9441afa4523c1a7a5341e3daae90e4c2a4e
Comment 9 Zac Medico gentoo-dev 2010-09-04 07:52:15 UTC
This is fixed in 2.1.9.
Comment 10 Zdenek Behan 2010-09-06 05:04:38 UTC
Thanks! Also, note that there is a bug open for something similar, which you probably might want to close too:

What should noted, that pbzip2 (the reason for this bug) is having slight trouble with the abuse that portage is giving it, in particular attaching bzipped garbage at the end of the binary packages.
I have a patch to at least make it stop failing, and beyond that, it seems to have no impact except for some of the unbzipped files being slightly larger (but tar seems to ignore that part). I'll investigate in more detail, and try to push the patches upstream to make it work 100% with portage.

Other than that, pbzip seems to be an entirely drop-in replacement, because it even supports identical options, and since v1.1.1 does not seem to face bunzip regression when switching from pbzip to bzip and back.
Comment 11 Zac Medico gentoo-dev 2010-09-06 05:55:03 UTC
*** Bug 264258 has been marked as a duplicate of this bug. ***