Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 248089 - compression support for portage 2.2 binary packages index file ('Packages')
Summary: compression support for portage 2.2 binary packages index file ('Packages')
Status: CONFIRMED
Alias: None
Product: Portage Development
Classification: Unclassified
Component: Binary packages support (show other bugs)
Hardware: All Linux
: High enhancement (vote)
Assignee: Portage team
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2008-11-21 23:45 UTC by Wolfram Schlich (RETIRED)
Modified: 2022-12-26 07:28 UTC (History)
2 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Wolfram Schlich (RETIRED) gentoo-dev 2008-11-21 23:45:51 UTC
downloading a 500kb (uncompressed) Packages file from
the binhost over a somewhat slow connection (UMTS in my case)
is a PITA, so can we please add compression support to the
code handling that file (creation on the binhost and
handling on the client)?
Comment 1 Zac Medico gentoo-dev 2008-11-22 08:35:32 UTC
How about if you configure your webserver for gzip compression and we add support to portage for that? I found some sample code that we can use here:

http://diveintopython.org/http_web_services/gzip_compression.html
Comment 2 Wolfram Schlich (RETIRED) gentoo-dev 2008-11-22 13:05:45 UTC
Well, that would be a start, but remember that portage also supports FTP
for fetching stuff from the binhost... wouldn't it be nice if that
compression would be transfer protocol agnostic?
Comment 3 Zac Medico gentoo-dev 2008-11-22 19:23:42 UTC
I suppose we can have portage create a Packages.gz and download that one instead of the uncompressed version.
Comment 4 Fabian Groffen gentoo-dev 2008-11-22 19:28:22 UTC
would make more sense to use bzip2 instead, since the packages are compressed with the same algorithm
Comment 5 Wolfram Schlich (RETIRED) gentoo-dev 2008-11-23 11:39:12 UTC
(In reply to comment #4)
> would make more sense to use bzip2 instead, since the packages are compressed
> with the same algorithm

Shouldn't the algorithm for both (tarballs and Packages) be
configurable anyway?! :) Something line PORTAGE_COMPRESS...
Could be named PORTAGE_BINHOST_COMPRESS and would need to
be specified on both the binhost and the clients in order
to use a specific compression method.
It could default to gzip though.
What do you think?
Comment 6 Fabian Groffen gentoo-dev 2008-11-23 11:43:47 UTC
(In reply to comment #5)
> It could default to gzip though.
> What do you think?

lzma, because it is compress once, decompress many times, IMO.  And lzma is a much better compressor too.

A configurable compressor would be pretty nice but causes some problems in distribution (think of hosts not having the right decompressor).  However, defaulting to gzip sounds unreasonable to me, since Gentoo has been using bzip2 for ages, and it is a better compressor than gzip.  And pbzip2 is nice too...:wq
Comment 7 Wolfram Schlich (RETIRED) gentoo-dev 2009-02-13 14:40:40 UTC
Any news on this one? :) TIA!