Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 525432 - www-client/firefox-24.8.1 package FTP fetching not working through Mozilla CDN
Summary: www-client/firefox-24.8.1 package FTP fetching not working through Mozilla CDN
Status: RESOLVED INVALID
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Current packages (show other bugs)
Hardware: All Linux
: Normal normal (vote)
Assignee: Gentoo Linux bug wranglers
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2014-10-14 19:25 UTC by fcrvincent
Modified: 2014-10-15 08:36 UTC (History)
0 users

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description fcrvincent 2014-10-14 19:25:28 UTC
emerge firefox-bin invariably leads to the following error:

  Logging in as anonymous ... Logged in!
  ==> SYST ... done.    ==> PWD ... done.
  ==> TYPE I ... done.  ==> CWD (1) /pub/mozilla.org/firefox/releases//24.8.1esr/linux-i686/xpi ... done.
  ==> SIZE en-GB.xpi ... 318297
  ==> PASV ... 
  Error in server response, closing control connection.
  Retrying.

The same applies to thunderbird-bin. 

I could not emerge those two packages over the past 2 months - it used to work flawlessly before that - and had to exclude the two packages from my emerge world in order to be able to update my Gentoo system.

It could be that Mozilla's Content Distribution Network does not support FTP well outside the USA (I am based in Europe). This issues seems to be similar to bug 439610. 

I found that using wget with the exact same URI's from the related ebuilds solves the problem completely. 

Is it possible for you to replace the MOZ_FTP_URI with a regular SRC_URI in the firefox-bin and thunderbird-bin ebuilds?


Many thanks in advance
Comment 1 fcrvincent 2014-10-14 19:55:05 UTC
I found that the firefox/thunderbird source packages employ the regular wget fetching method and people seem happy with this - not sure why the pre-compiled versions can not be fetched through the same method?

Emerging the source packages of firefox/thunderbird is of course a work-around but it takes a long time to download and compile.

Hopefully you can find a solution to this failed FTP fetching issue. 

Again: 
ftp ftp.mozilla.org/pub/mozilla.org/firefox/releases//24.8.1esr/linux-i686/xpi/en-GB.xpi fails

but 
wget ftp.mozilla.org/pub/mozilla.org/firefox/releases//24.8.1esr/linux-i686/xpi/en-GB.xpi works perfectly.
Comment 2 Alexander Tsoy 2014-10-14 22:04:27 UTC
(In reply to fcrvincent from comment #1)

> Again: 
> ftp
> ftp.mozilla.org/pub/mozilla.org/firefox/releases//24.8.1esr/linux-i686/xpi/
> en-GB.xpi fails
> 
> but 
> wget
> ftp.mozilla.org/pub/mozilla.org/firefox/releases//24.8.1esr/linux-i686/xpi/
> en-GB.xpi works perfectly.

Err.. wget is used by default:

$ grep FETCHCOMMAND= /usr/share/portage/config/make.globals
FETCHCOMMAND="wget -t 3 -T 60 --passive-ftp -O \"\${DISTDIR}/\${FILE}\" \"\${URI}\""
Comment 3 fcrvincent 2014-10-15 07:02:17 UTC
I see the problem now:

command 'wget -t 3 -T 60 --passive-ftp'+URI defaults to passive ftp on my system not to a normal http wget?? 

This is what emerge -v firefox-bin gives me:

emerge -v firefox-bin

These are the packages that would be merged, in order:

Calculating dependencies... done!
[ebuild     U  ] www-client/firefox-bin-24.8.1 [24.7.0] USE="startup-notification (-selinux)" LINGUAS="en_GB -af -ak -ar -as -ast -be -bg -bn_BD -bn_IN -br -bs -ca -cs -csb -cy -da -de -el -en_ZA -eo -es_AR -es_CL -es_ES -es_MX -et -eu -fa -fi -fr -fy_NL -ga_IE -gd -gl -gu_IN -he -hi_IN -hr -hu -hy_AM -id -is -it -ja -kk -kn -ko -ku -lg -lt -lv -mai -mk -ml -mr -nb_NO -nl -nn_NO -nso -or -pa_IN -pl -pt_BR -pt_PT -rm -ro -ru -si -sk -sl -son -sq -sr -sv_SE -ta -ta_LK -te -th -tr -uk -vi -zh_CN -zh_TW -zu" 28,570 kB

Total: 1 package (1 upgrade), Size of downloads: 28,570 kB


>>> Verifying ebuild manifests

>>> Emerging (1 of 1) www-client/firefox-bin-24.8.1
>>> Downloading 'ftp://ftp.mozilla.org/pub/mozilla.org/firefox/releases//24.8.1esr/linux-i686/xpi/en-GB.xpi'
--2014-10-15 08:01:06--  ftp://ftp.mozilla.org/pub/mozilla.org/firefox/releases//24.8.1esr/linux-i686/xpi/en-GB.xpi
           => ‘/usr/portage/distfiles/firefox-24.8.1esr-en-GB.xpi’
Resolving ftp.mozilla.org... 63.245.215.46, 63.245.215.56
Connecting to ftp.mozilla.org|63.245.215.46|:21... connected.
Logging in as anonymous ... Logged in!
==> SYST ... done.    ==> PWD ... done.
==> TYPE I ... done.  ==> CWD (1) /pub/mozilla.org/firefox/releases//24.8.1esr/linux-i686/xpi ... done.
==> SIZE en-GB.xpi ... 318297
==> PASV ... 
Error in server response, closing control connection.
Retrying.
Comment 4 fcrvincent 2014-10-15 07:23:12 UTC
Also if this can help you spot the issue the emerge of firefox-bin and thunderbird-bin used to work until 24.7.0 

And all other packages on my system have no problem with emerge updating/fetching.
Comment 5 fcrvincent 2014-10-15 07:30:43 UTC
The problem is that URI  ftp://ftp.mozilla.org/pub/mozilla.org/firefox/releases//24.8.1esr/linux-i686/xpi/en-GB.xpi forces an FTP download

You should use  ftp.mozilla.org/pub/mozilla.org/firefox/releases//24.8.1esr/linux-i686/xpi/en-GB.xpi without the protocol specifier instead
Comment 6 fcrvincent 2014-10-15 07:44:36 UTC
Just tried 

wget http://ftp.mozilla.org/pub/mozilla.org/firefox/releases//24.8.1esr/linux-i686/xpi/en-GB.xpi

and 

wget ftp.mozilla.org/pub/mozilla.org/firefox/releases//24.8.1esr/linux-i686/xpi/en-GB.xpi

From a US (ec2) server and both work. 

The FTP download from the US ec2 instance also works. 
However it does not work from where I am - probably it is a CDN related issue.

Why not use HTTP that works in all cases instead of FTP that could be failing everywhere but in the States?
Comment 7 Alexander Tsoy 2014-10-15 07:46:01 UTC
(In reply to fcrvincent from comment #4)

> And all other packages on my system have no problem with emerge
> updating/fetching.

That's because you have http mirror first in GENTOO_MIRRORS variable. firefox-bin distfiles gets downloaded from mozilla ftp because gentoo cannot mirror them due to legal issues.

You probably have problems with firewall on your side, or your ISP's side. If ftp in active mode works for you, then copy FETCHCOMMAND from /usr/share/portage/config/make.globals to /etc/portage/make.conf and replace --passive-ftp with --no-passive-ftp.
Comment 8 fcrvincent 2014-10-15 07:49:43 UTC
Also sticking to the FTP protocol that does not work universally could dissuade some Gentoo existing or new users to stay with Gentoo. 

I am the second to report this issue and it is likely that others have experienced the same but are not willing or do not how to report the issue.
Comment 9 fcrvincent 2014-10-15 07:58:41 UTC
(In reply to Alexander Tsoy from comment #7)
> (In reply to fcrvincent from comment #4)
> 
> > And all other packages on my system have no problem with emerge
> > updating/fetching.
> 
> That's because you have http mirror first in GENTOO_MIRRORS variable.
> firefox-bin distfiles gets downloaded from mozilla ftp because gentoo cannot
> mirror them due to legal issues.
> 
> You probably have problems with firewall on your side, or your ISP's side.
> If ftp in active mode works for you, then copy FETCHCOMMAND from
> /usr/share/portage/config/make.globals to /etc/portage/make.conf and replace
> --passive-ftp with --no-passive-ftp.


Why not use http:// instead of ftp:// ? It also works with Mozilla's servers from my US based ec2 instance??
Comment 10 fcrvincent 2014-10-15 08:09:30 UTC
(In reply to Alexander Tsoy from comment #7)
> (In reply to fcrvincent from comment #4)
> 
> > And all other packages on my system have no problem with emerge
> > updating/fetching.
> 
> That's because you have http mirror first in GENTOO_MIRRORS variable.
> firefox-bin distfiles gets downloaded from mozilla ftp because gentoo cannot
> mirror them due to legal issues.
> 
> You probably have problems with firewall on your side, or your ISP's side.
> If ftp in active mode works for you, then copy FETCHCOMMAND from
> /usr/share/portage/config/make.globals to /etc/portage/make.conf and replace
> --passive-ftp with --no-passive-ftp.

My firewall rules are empty and I will check if I have problems with passive FTP through my ISP however fetching firefox-bin and thunderbird-bin until 24.7.0 worked flawlessly.

After that either my ISP suddenly firewalled passive FTP or the firefox/thunderbird-bin ebuilds were changed from HTTP download to FTP download?
Comment 11 fcrvincent 2014-10-15 08:15:15 UTC
(In reply to Alexander Tsoy from comment #7)
> (In reply to fcrvincent from comment #4)
> 
> > And all other packages on my system have no problem with emerge
> > updating/fetching.
> 
> That's because you have http mirror first in GENTOO_MIRRORS variable.
> firefox-bin distfiles gets downloaded from mozilla ftp because gentoo cannot
> mirror them due to legal issues.
> 
> You probably have problems with firewall on your side, or your ISP's side.
> If ftp in active mode works for you, then copy FETCHCOMMAND from
> /usr/share/portage/config/make.globals to /etc/portage/make.conf and replace
> --passive-ftp with --no-passive-ftp.

My US-based ec2 server does not run on Gentoo and the command wget http://ftp.mozilla.org/pub/mozilla.org/firefox/releases//24.8.1esr/linux-i686/xpi/en-GB.xpi works so not sure if this has to do with my GENTOO_MIRRORS variable.
Comment 12 fcrvincent 2014-10-15 08:26:04 UTC
(In reply to fcrvincent from comment #10)
> (In reply to Alexander Tsoy from comment #7)
> > (In reply to fcrvincent from comment #4)
> > 
> > > And all other packages on my system have no problem with emerge
> > > updating/fetching.
> > 
> > That's because you have http mirror first in GENTOO_MIRRORS variable.
> > firefox-bin distfiles gets downloaded from mozilla ftp because gentoo cannot
> > mirror them due to legal issues.
> > 
> > You probably have problems with firewall on your side, or your ISP's side.
> > If ftp in active mode works for you, then copy FETCHCOMMAND from
> > /usr/share/portage/config/make.globals to /etc/portage/make.conf 

The --no-passive-ftp FETCHCOMMAND alteration in the make.conf works for now. 

Shouldn't all users be at least warned in an eselect news bulletin of this?
Comment 13 fcrvincent 2014-10-15 08:34:17 UTC
(In reply to Alexander Tsoy from comment #7)
> (In reply to fcrvincent from comment #4)
> 
> > And all other packages on my system have no problem with emerge
> > updating/fetching.
> 
> That's because you have http mirror first in GENTOO_MIRRORS variable.
> firefox-bin distfiles gets downloaded from mozilla ftp because gentoo canot
> mirror them due to legal issues.
> 
> You probably have problems with firewall on your side, or your ISP's side.
> If ftp in active mode works for you, then copy FETCHCOMMAND from
> /usr/share/portage/config/make.globals to /etc/portage/make.conf and replace
> --passive-ftp with --no-passive-ftp.

Just tried a PASV FTP download on an anonymous test server and it worked - so my firewall / ISP is not the issue. 

The issue seems to be with the Mozilla's Content Distribution Network that does not support passive ftp for some reason.
Comment 14 Jeroen Roovers (RETIRED) gentoo-dev 2014-10-15 08:36:14 UTC
Please take this discussion to the forums.