Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 39407 - "-z" (compression) during 'emerge sync' should be optional.
Summary: "-z" (compression) during 'emerge sync' should be optional.
Status: RESOLVED LATER
Alias: None
Product: Portage Development
Classification: Unclassified
Component: Conceptual/Abstract Ideas (show other bugs)
Hardware: All Linux
: High enhancement (vote)
Assignee: Mirror Admins
URL: $ rsync ftp.rhnet.is::
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2004-01-25 18:10 UTC by Ævar Arnfjörð Bjarmason
Modified: 2004-04-14 18:27 UTC (History)
2 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Ævar Arnfjörð Bjarmason 2004-01-25 18:10:21 UTC
Some rsync servers such as ftp.rhnet.is specifically ask people not to use compression during syncs. This behavior is however hardcoded into portage and cannot be changed save for editing the python source code.

I suggest either a :
SYNC_COMPRESSION="[yes|no,1|0]" option in /etc/make.conf to control this behavior or
SYNC_OPTIONS="-rlptDvz --progress --stats --delete --delete-after --timeout="+str(mytimeout)+" --exclude='distfiles/*' --exclude='local/*' --exclude='packages/*'" so other people could specifically tune emerge sync to their needs (which might be this)

Reference:
http://www.mail-archive.com/gentoo-dev@gentoo.org/msg02204.html (and following threads)

Note: i am not the administrator of the mirror in question.


Reproducible: Always
Steps to Reproduce:
Comment 1 Benjamin Coles 2004-01-26 14:14:18 UTC
not my bug, sorry.
Comment 2 Corey Shields 2004-02-08 12:06:15 UTC
sending this bug to the portage team....
Comment 3 SpanKY gentoo-dev 2004-02-08 12:17:03 UTC
this is something that needs to be decided by the mirror admins and/or discussed with our mirrors
Comment 4 Corey Shields 2004-02-08 12:33:37 UTC
Well if we want to take the mirror perspective then it should be done away with because it does add a lot of load to the server.  However compression helps those with slow connections.  The reason I want to leave it up to the portage team is because there had to be some reason for them to code the -z in the first place.

As for the mirrors, there is nothing that says they should specifically allow compression, but on the other hand there is nothing that says they can not deny it[1].  If they deny it then that does break the portage connection and they really don't do us much good, so either this needs to become a feature implementation in portage, or a new policy to implement for the rsync mirrors.  I would opt for the latter, because from the user's perspective there is no real good way to tell whether their selected mirror or round robin set supports compression or not.   And things have worked fine with compression thus far.

[1] - http://www.gentoo.org/doc/en/rsync.xml

Cheers,

-Corey
Comment 5 Kurt Lieber (RETIRED) gentoo-dev 2004-04-14 18:27:47 UTC
As Corey pointed out, we don't specifically forbid the use of compression.  We have, however, had unpredictable results when using compression, especially when the two sides are using different versions of rsync.

This might be a future feature request, but it would require a great deal of testing beforehand to ensure that things work properly. If it were to be enabled, adopted widely and then proved to be problematic, that could be disastrous for us.

While we may look at this in the future, it is not a priority for us now as there are many other things that take precedence over this. As such, closing as later.  (And it should be treated as "Later (maybe)")