Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 611072 - net-misc/wget[-ssl] errors out for https://
Summary: net-misc/wget[-ssl] errors out for https://
Status: RESOLVED FIXED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Current packages (show other bugs)
Hardware: All Linux
: Normal normal
Assignee: Gentoo's Team for Core System packages
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2017-02-27 09:17 UTC by Toralf Förster
Modified: 2020-03-28 07:17 UTC (History)
1 user (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
emerge-info.txt (emerge-info.txt,5.58 KB, text/plain)
2017-02-27 09:17 UTC, Toralf Förster
Details
dev-python:os-client-config-1.26.0:20170227-091317.log (dev-python:os-client-config-1.26.0:20170227-091317.log,6.33 KB, text/plain)
2017-02-27 09:17 UTC, Toralf Förster
Details
emerge-history.txt (emerge-history.txt,365.63 KB, text/plain)
2017-02-27 09:17 UTC, Toralf Förster
Details
etc.portage.tbz2 (etc.portage.tbz2,36.94 KB, application/x-bzip)
2017-02-27 09:17 UTC, Toralf Förster
Details
find_ebuilds_using_https_in_SRC_URI_and_mirror_in_RESTRICT (find_ebuilds_using_https_in_SRC_URI_and_mirror_in_RESTRICT,1.01 KB, text/plain)
2017-02-28 08:29 UTC, Arfrever Frehtes Taifersar Arahesis
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Toralf Förster gentoo-dev 2017-02-27 09:17:23 UTC
>>> Downloading 'https://files.pythonhosted.org/packages/source/o/os-client-config/os-client-config-1.26.0.tar.gz'
https://files.pythonhosted.org/packages/source/o/os-client-config/os-client-config-1.26.0.tar.gz: HTTPS support not compiled in.
!!! Couldn't download 'os-client-config-1.26.0.tar.gz'. Aborting.
 * Fetch failed for 'dev-python/os-client-config-1.26.0', Log file:
 *  '/var/log/portage/dev-python:os-client-config-1.26.0:20170227-091317.log'

  -----------------------------------------------------------------

  This is an unstable amd64 chroot image (named hardened_20170217-144156) at a hardened host acting as a tinderbox.

  -----------------------------------------------------------------

gcc-config -l:
 [1] x86_64-pc-linux-gnu-5.4.0 *
 [2] x86_64-pc-linux-gnu-5.4.0-hardenednopie
 [3] x86_64-pc-linux-gnu-5.4.0-hardenednopiessp
 [4] x86_64-pc-linux-gnu-5.4.0-hardenednossp
 [5] x86_64-pc-linux-gnu-5.4.0-vanilla

Available Python interpreters, in order of preference:
  [1]   python3.4
  [2]   python3.6 (fallback)
  [3]   python3.5 (fallback)
  [4]   python2.7 (fallback)
Available Ruby profiles:
  [1]   ruby21 (with Rubygems) *
java-config:
The following VMs are available for generation-2:
Comment 1 Toralf Förster gentoo-dev 2017-02-27 09:17:26 UTC
Created attachment 465370 [details]
emerge-info.txt
Comment 2 Toralf Förster gentoo-dev 2017-02-27 09:17:29 UTC
Created attachment 465372 [details]
dev-python:os-client-config-1.26.0:20170227-091317.log
Comment 3 Toralf Förster gentoo-dev 2017-02-27 09:17:32 UTC
Created attachment 465374 [details]
emerge-history.txt
Comment 4 Toralf Förster gentoo-dev 2017-02-27 09:17:35 UTC
Created attachment 465376 [details]
etc.portage.tbz2
Comment 5 Toralf Förster gentoo-dev 2017-02-27 09:27:20 UTC
ick, this is the wget -ssl error
Comment 6 Arfrever Frehtes Taifersar Arahesis 2017-02-27 19:05:24 UTC
Maybe "net-misc/wget ssl" entry in package.use.force would be reasonable.
Comment 7 SpanKY gentoo-dev 2017-02-27 20:59:11 UTC
there's a lot of USE settings in the tree that are not recommended and will break things if you disable them, but we don't hold people's hands and force them to go with specific settings.  i think this falls into that bucket.

setting USE=-ssl on wget and then trying to use https:// URIs makes no sense and is an operator error
Comment 8 Arfrever Frehtes Taifersar Arahesis 2017-02-28 07:57:40 UTC
I was suggesting entry in package.use.force not due to users directly using wget for https://, but due to ${repository_gentoo}/profiles/thirdpartymirrors containing some lines with only https:// URLs:

github    https://github.com/downloads
pypi      https://pypi.python.org/packages/source https://files.pythonhosted.org/packages/source
rubygems  https://rubygems.org/gems
Comment 9 Arfrever Frehtes Taifersar Arahesis 2017-02-28 08:29:54 UTC
Created attachment 465494 [details]
find_ebuilds_using_https_in_SRC_URI_and_mirror_in_RESTRICT

This script shows for me:

Found packages:        77
Found package-slots:   81
Found ebuilds:         145
...
Comment 10 SpanKY gentoo-dev 2017-02-28 15:17:09 UTC
(In reply to Arfrever Frehtes Taifersar Arahesis from comment #9)

this assumes wget is used as the fetching tool, and the system is compiling packages directly on the device.  there are valid scenarios, albeit probably uncommon, where those situations don't occur.

this build scenario has existed for over 10 years at this point and this is the first bug that i can remember coming up.  or at least, it hasn't been frequent enough for me to remember.  if people want to build setups that might result in a broken setup, then they're free to do so.
Comment 11 Matt Turner gentoo-dev 2020-03-28 03:27:40 UTC
(In reply to SpanKY from comment #10)
> (In reply to Arfrever Frehtes Taifersar Arahesis from comment #9)
> 
> this assumes wget is used as the fetching tool, and the system is compiling
> packages directly on the device.  there are valid scenarios, albeit probably
> uncommon, where those situations don't occur.
> 
> this build scenario has existed for over 10 years at this point and this is
> the first bug that i can remember coming up.  or at least, it hasn't been
> frequent enough for me to remember.  if people want to build setups that
> might result in a broken setup, then they're free to do so.

I accept that there are niche cases where you may want to avoid the gnutls/openssl dependency.

At the same time, this is an extreme niche case.

Unfortunately the issue Toralf reports has and does continue to reoccur:

https://forums.gentoo.org/viewtopic-p-8162980.html

I just hit it myself doing a stage build. I think a completely fair compromise is to package.use.force it on, given that wget is the default FETCHCOMMAND in portage and probably the vast majority of the porttree's SRC_URIs are https these days.

I'll verify that doesn't cause problems with stage builds and then proceed.
Comment 12 Mike Gilbert gentoo-dev 2020-03-28 04:01:18 UTC
I don't think this warrants a package.use.force entry at all.

The only people affected would be those that set USE="-*" or USE="-ssl", both of which are incredibly stupid if you don't know what you're doing.

A forum thread that gets a reply once a year is hardly compelling evidence that a significant number of users are affected by this.
Comment 13 Mike Gilbert gentoo-dev 2020-03-28 04:11:56 UTC
After discussion in IRC, I withdraw my objection.
Comment 14 Larry the Git Cow gentoo-dev 2020-03-28 07:09:45 UTC
The bug has been closed via the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=d141380b915dc0896bebacfa835b6400cdd90ac3

commit d141380b915dc0896bebacfa835b6400cdd90ac3
Author:     Matt Turner <mattst88@gentoo.org>
AuthorDate: 2020-03-28 07:08:28 +0000
Commit:     Matt Turner <mattst88@gentoo.org>
CommitDate: 2020-03-28 07:09:36 +0000

    profiles: Force USE=ssl for net-misc/wget
    
    Closes: https://bugs.gentoo.org/611072
    Signed-off-by: Matt Turner <mattst88@gentoo.org>

 profiles/base/package.use.force | 5 +++++
 1 file changed, 5 insertions(+)
Comment 15 Matt Turner gentoo-dev 2020-03-28 07:17:18 UTC
Some explanation. I hit this with a catalyst stage build. I believe that the possibility for this failure has existed for many years, but it is very rare because it requires:

(1) catalyst stage2/stage3 needs to download a distfile, which is rare since most systems have those distfiles already
(2) there's no binpkg available, otherwise it wouldn't need the distfile
(3) the SRC_URI of the package is HTTPS-only
(4) portage has to try to download the distfile before wget is reemerged with USE=ssl

Point (4) is particularly irritating, because it's not really deterministic between catalyst runs. Case in point, I re-ran catalyst immediately after the failure and portage picked a different order to install packages which lead to wget[ssl] being available by the time the HTTPS-only distfile was downloaded.


Summarizing the discussion on IRC:

It's a rare failure and often self-inflicted by USE="-*". Catalyst stage1 does this in fact, which is not ideal but I don't have a better solution in mind.

Also, there's not a particularly good argument for allowing wget[-ssl] except some strange niches, in which case just unforce it. Unclear what a system without openssl is capable of doing though...