Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 636238 - sys-apps/portage-2.3.8: fetched tar.gz archives are uncompressed automatically by net-misc/wget-1.19.2
Summary: sys-apps/portage-2.3.8: fetched tar.gz archives are uncompressed automaticall...
Alias: None
Product: Portage Development
Classification: Unclassified
Component: Core - External Interaction (show other bugs)
Hardware: All All
: Normal normal (vote)
Assignee: Portage team
Depends on:
Reported: 2017-11-02 05:15 UTC by Heiko Schäfer
Modified: 2017-12-13 21:17 UTC (History)
2 users (show)

See Also:
Package list:
Runtime testing required: ---


Note You need to log in before you can comment on or make changes to this bug.
Description Heiko Schäfer 2017-11-02 05:15:32 UTC
Fetching tar.gz archives from at least are getting uncompressed after download by wget. the http-protocol (i.e. does not show this behaviour. This results in checksum mismatches and therefore in a package refetch.

I got this problem solved by adding


in make.conf in the variables FETCHCOMMAND and RESUMECOMMAND

Reproducible: Always

Steps to Reproduce:
emerge --nodeps --fetchonly dev-java/protobuf-java
Comment 1 Mike Gilbert gentoo-dev 2017-11-02 14:38:20 UTC
This sounds like a bug in wget. Decompressing files from FTP automatically is not a "feature".

The --compression options doesn't mention doing anything for ftp servers; it only mentions the Content-Encoding HTTP header.
Comment 2 Mike Gilbert gentoo-dev 2017-11-02 14:46:17 UTC
I am unable to reproduce this with net-misc/wget-1.19.2.

% GENTOO_MIRRORS="" ebuild xz-utils-5.2.3.ebuild fetch
>>> Downloading ''
--2017-11-02 10:44:11--
           => ‘/var/portage/distfiles/xz-5.2.3.tar.gz’
Resolving (, 2001:638:208:ef1b:0:ff:fe00:4
Connecting to (||:21... connected.
Logging in as anonymous ... Logged in!
==> SYST ... done.    ==> PWD ... done.
==> TYPE I ... done.  ==> CWD (1) /pub/linux/gentoo/distfiles ... done.
==> SIZE xz-5.2.3.tar.gz ... 1490665
==> PASV ... done.    ==> RETR xz-5.2.3.tar.gz ... done.
Length: 1490665 (1.4M) (unauthoritative)

xz-5.2.3.tar.gz     100%[===================>]   1.42M  1.93MB/s    in 0.7s

2017-11-02 10:44:13 (1.93 MB/s) - ‘/var/portage/distfiles/xz-5.2.3.tar.gz’ saved [1490665]

 * xz-5.2.3.tar.gz SHA256 SHA512 WHIRLPOOL size ;-) ...                  [ ok ]

% file /var/portage/distfiles/xz-5.2.3.tar.gz
/var/portage/distfiles/xz-5.2.3.tar.gz: gzip compressed data, max compression, from Unix
Comment 3 Heiko Schäfer 2017-11-02 15:07:13 UTC
(In reply to Mike Gilbert from comment #2)
> I am unable to reproduce this with net-misc/wget-1.19.2.

I found the error: my squid proxy uncompressed the archive. So you can close this bug as invalid.

It is strange to me, this happened so unexpected. I haven't touched my proxy config for months, but the error just appeared first in the last days.
Comment 4 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2017-12-07 21:37:28 UTC
Just FYI, this is not a bug in wget, though the description might be confusing. The option is to enable transparent compression of files that are not compressed. The server must not use this for download of compressed files, since it does not compress them transparently (well, unless it does second layer of compression).

The question then is: did you actually set anything non-standard in Squid that caused this? If not, then it's a bug in Squid and it should be reported as such.
Comment 5 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2017-12-13 21:17:13 UTC
*** Bug 640930 has been marked as a duplicate of this bug. ***