Bugzilla Bug 163782

Fetching emails from my ISP using fetchmail has started to fail since version
6.3.6 (the previous version I had installed, 6.3.4, had no such problem). This
is the output when using verbose mode while trying to check mails:

# fetchmail -v --check -N -f /etc/fetchmailrc 
fetchmail: WARNING: Running as root is discouraged.
fetchmail: 6.3.6 querying gye.satnet.net (protocol POP3) at Thu Jan 25 12:23:05
2007: poll started
Trying to connect to XXX.XXX.XXX.XXX/110...connected.
fetchmail: POP3< +OK Hello there.
fetchmail: POP3> CAPA
fetchmail: POP3< +OK Here's what I can do:
fetchmail: POP3< STLS
fetchmail: POP3< TOP
fetchmail: POP3< USER
fetchmail: POP3< LOGIN-DELAY 10
fetchmail: POP3< PIPELINING
fetchmail: POP3< UIDL
fetchmail: POP3< IMPLEMENTATION Courier Mail Server
fetchmail: POP3< .
fetchmail: POP3> STLS
fetchmail: POP3< +OK Begin SSL/TLS negotiation now.
fetchmail: gye.satnet.net: opportunistic upgrade to TLS failed, trying to
continue.
fetchmail: POP3> USER <username>
fetchmail: Unknown login or authentication error on <username>@gye.satnet.net
fetchmail: socket error while fetching from <username>@gye.satnet.net

If I force no SSL, it works fine, though. Note that if I use version 6.3.4, I
have no problems using SSL mode:

# fetchmail -v --check -N -f /etc/fetchmailrc 
fetchmail: WARNING: Running as root is discouraged.
fetchmail: 6.3.4 querying gye.satnet.net (protocol POP3) at Thu Jan 25 12:32:05
2007: poll started
fetchmail: POP3< +OK Hello there.
fetchmail: POP3> CAPA
fetchmail: POP3< +OK Here's what I can do:
fetchmail: POP3< STLS
fetchmail: POP3< TOP
fetchmail: POP3< USER
fetchmail: POP3< LOGIN-DELAY 10
fetchmail: POP3< PIPELINING
fetchmail: POP3< UIDL
fetchmail: POP3< IMPLEMENTATION Courier Mail Server
fetchmail: POP3< .
fetchmail: POP3> STLS
fetchmail: POP3< +OK Begin SSL/TLS negotiation now.
fetchmail: Repoll immediately on <username>@gye.satnet.net
fetchmail: POP3< +OK Hello there.
fetchmail: POP3> CAPA
fetchmail: POP3< +OK Here's what I can do:
fetchmail: POP3< STLS
fetchmail: POP3< TOP
fetchmail: POP3< USER
fetchmail: POP3< LOGIN-DELAY 10
fetchmail: POP3< PIPELINING
fetchmail: POP3< UIDL
fetchmail: POP3< IMPLEMENTATION Courier Mail Server
fetchmail: POP3< .
fetchmail: POP3> USER <username>
fetchmail: POP3< +OK Password required.
fetchmail: POP3> PASS *
fetchmail: POP3< +OK logged in.
fetchmail: POP3> STAT
fetchmail: POP3< +OK 0 0
fetchmail: No mail for <username> at gye.satnet.net
fetchmail: POP3> QUIT

My other email account doesn't use SSL, so I can't test if the problem is
specific to my ISP or others can trigger it too. I would had posted this to
fetchmail directly, but I got lost on their page finding the bugtracker...

Reproducible: Always

Steps to Reproduce:
1. Do a "fetchmail -v --check -N -f /etc/fetchmailrc" that checks against a ISP
that supports SSL and triggers the problem (such as gye.satnet.net)
Actual Results:  
SSL negotiation failed, afterwards the USER command gives an invalid username
error followed by a socket error.

Expected Results:  
SSL negotiation should be successful, as well as the USER command and all
following ones.

This happens both on the machine at home and my machine at work (one is an AMD
AthlonXP, the other is a Pentium III older Dell). Both are using Gentoo stable
(x86).

This is the emerge info of my main machine:
# emerge --info
Portage 2.1.1-r2 (default-linux/x86/2006.0, gcc-4.1.1, glibc-2.4-r4,
2.6.17-reiser4-r7 i686)
=================================================================
System uname: 2.6.17-reiser4-r7 i686 AMD Athlon(tm)
Gentoo Base System version 1.12.6
Last Sync: Sat, 20 Jan 2007 08:50:01 +0000
app-admin/eselect-compiler: [Not Present]
dev-java/java-config: 1.3.7, 2.0.31
dev-lang/python:     2.3.3-r1, 2.4.3-r4
dev-python/pycrypto: 2.0.1-r5
dev-util/ccache:     [Not Present]
dev-util/confcache:  [Not Present]
sys-apps/sandbox:    1.2.17
sys-devel/autoconf:  2.13, 2.61
sys-devel/automake:  1.4_p6, 1.5, 1.6.3, 1.7.9-r1, 1.8.5-r3, 1.9.6-r2, 1.10
sys-devel/binutils:  2.16.1-r3
sys-devel/gcc-config: 1.3.14
sys-devel/libtool:   1.4.3-r4, 1.5.22
virtual/os-headers:  2.6.17-r2
ACCEPT_KEYWORDS="x86"
AUTOCLEAN="yes"
CBUILD="i686-pc-linux-gnu"
CFLAGS="-march=athlon-xp -O2 -fomit-frame-pointer -momit-leaf-frame-pointer
-fno-ident -pipe"
CHOST="i686-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/share/X11/xkb"
CONFIG_PROTECT_MASK="/etc/env.d /etc/env.d/java/ /etc/gconf
/etc/java-config/vms/ /etc/revdep-rebuild /etc/splash /etc/terminfo"
CXXFLAGS="-O2 -mcpu=i686 -pipe"
DISTDIR="/usr/portage/distfiles"
FEATURES="autoconfig digest distlocks metadata-transfer parallel-fetch sandbox
sfperms strict userfetch userpriv"
GENTOO_MIRRORS="http://gentoo.mirrors.tds.net/gentoo
http://ftp.du.se/pub/os/gentoo http://mirrors.acm.cs.rpi.edu/gentoo/
http://gentoo.ccccom.com"
MAKEOPTS="-j2 -s"
PKGDIR="/usr/portage/packages"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress
--force --whole-file --delete --delete-after --stats --timeout=180
--exclude='/distfiles' --exclude='/local' --exclude='/packages'"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
PORTDIR_OVERLAY="/etc/portage/overlay"
SYNC="rsync://rsync.samerica.gentoo.org/gentoo-portage"
USE="x86 3dnow X aac acpi alsa alsa_cards_ali5451 alsa_cards_als4000
alsa_cards_atiixp alsa_cards_atiixp-modem alsa_cards_bt87x alsa_cards_ca0106
alsa_cards_cmipci alsa_cards_emu10k1x alsa_cards_ens1370 alsa_cards_ens1371
alsa_cards_es1938 alsa_cards_es1968 alsa_cards_fm801 alsa_cards_hda-intel
alsa_cards_intel8x0 alsa_cards_intel8x0m alsa_cards_maestro3 alsa_cards_trident
alsa_cards_usb-audio alsa_cards_via82xx alsa_cards_via82xx-modem
alsa_cards_ymfpci alsa_pcm_plugins_adpcm alsa_pcm_plugins_alaw
alsa_pcm_plugins_asym alsa_pcm_plugins_copy alsa_pcm_plugins_dmix
alsa_pcm_plugins_dshare alsa_pcm_plugins_dsnoop alsa_pcm_plugins_empty
alsa_pcm_plugins_extplug alsa_pcm_plugins_file alsa_pcm_plugins_hooks
alsa_pcm_plugins_iec958 alsa_pcm_plugins_ioplug alsa_pcm_plugins_ladspa
alsa_pcm_plugins_lfloat alsa_pcm_plugins_linear alsa_pcm_plugins_meter
alsa_pcm_plugins_mulaw alsa_pcm_plugins_multi alsa_pcm_plugins_null
alsa_pcm_plugins_plug alsa_pcm_plugins_rate alsa_pcm_plugins_route
alsa_pcm_plugins_share alsa_pcm_plugins_shm alsa_pcm_plugins_softvol apm avi
berkdb bitmap-fonts boundschecking bzip2 cairo canna cjk cli cracklib crypt
cscope cups dbus dedicated dga divx4linux dlloader dri dts dvd dvdr dvdread
elibc_glibc emboss encode fam fbcon ffmpeg flac foomaticdb fortran freewnn gd
gdbm gif gimp ginac gpm gstreamer gtk gtk2 gtkhtml hal howl iconv imap imlib
innodb input_devices_evdev input_devices_keyboard input_devices_mouse
input_devices_wacom ipv6 isdnlog jabber java jikes jpeg kernel_linux
lcd_devices_bayrad lcd_devices_cfontz lcd_devices_cfontz633 lcd_devices_glk
lcd_devices_hd44780 lcd_devices_lb216 lcd_devices_lcdm001 lcd_devices_mtxorb
lcd_devices_ncurses lcd_devices_text libg++ libwww mad matroska mbox mikmod mmx
mng mp3 mpeg musepack ncurses nls nptl nvidia ogg opengl pam pcre pdf pdflib
perl png ppds pppd python quicktime readline reflection scanner sdl session
speex spell spl sse ssl svg svga tcltk tcpd theora threads tiff timidity
truetype truetype-fonts type1-fonts udev unicode usb userland_GNU v4l v4l2
video_cards_nvidia video_cards_vesa vorbis win32codecs wma wmf xface xine xml
xml2 xorg xv xvid zlib"
Unset:  CTARGET, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LANG, LC_ALL, LDFLAGS,
LINGUAS, PORTAGE_RSYNC_EXTRA_OPTS

Reported at upstream bug tracker:

https://developer.berlios.de/bugs/?func=detailbug&bug_id=10133&group_id=1824

Note that I got a different error with TLS negotiation.

Can you please post your .fetchmailrc entry for this server? (With obfuscated
password and/or username, of course)

The fetchmailrc file I use is quite simple:

# Configuration created Wed Sep 18 12:13:34 2002 by fetchmailconf
set postmaster "<my linux user>"
set bouncemail
set daemon 30
set no spambounce

poll gye.satnet.net
        proto pop3
        user "<username here>"
        pass "<password here>"
        is <linux username here>
        limit 0
        fetchall

As you can see, this seems like a very easy to reproduce problem. I just wonder
if something may be broken in the ISP rather than fetchmail... but if it worked
on the previous version, then it's more likely that something broke on the
latest version instead.

Same here.

fetchmail: POP3< +OK <17661.1170383361@svr306>
fetchmail: POP3> CAPA
fetchmail: POP3< +OK Capability list follows
fetchmail: POP3< TOP
fetchmail: POP3< UIDL
fetchmail: POP3< LAST
fetchmail: POP3< USER
fetchmail: POP3< APOP
fetchmail: POP3< STLS
fetchmail: POP3< .
fetchmail: POP3> STLS
fetchmail: POP3< +OK starting TLS negotiation
fetchmail: <hostname>: opportunistic upgrade to TLS failed, trying to continue
.
fetchmail: POP3> USER <username>
fetchmail: POP3< authorization first
27126:error:1408F10B:SSL routines:SSL3_GET_RECORD:wrong version
number:s3_pkt.c:
288:
fetchmail: Unknown login or authentication error on <username>@<hostname>
fetchmail: POP3> QUIT
fetchmail: POP3< -ERR authorization first
fetchmail: authorization first
fetchmail: client/server protocol error while fetching from
<username>@<hostname>
fetchmail: 6.3.6 querying <hostname> (protocol POP3) at Fri Feb  2 11:29:21 20
07: poll completed

Fetchmail upstream have admitted that there is a regression in 6.3.6 (see link
at comment #1), so until it's fixed in next release, please use previous
versions where it works.

This bug is supposed to be fixed in the upstream 6.3.7-rc1 version.
-- Matthias Andree, upstream fetchmail maintainer
(thanks for the Gentoo bug # in our upstream tracker)

Created an attachment (id=110451) [details]
patch to fix STLS regression

As there's already a release candidate for 6.3.7, I think we'll wait for 6.3.7
to come out instead of applying the patch.

Thanks for the info, Matthias!

6.3.7 is out, and has just entered Portage tree. Please test.

error occurred.

fetchmail: 6.3.7 querying <hostnameA> (protocol POP3) at Tue Feb 20 14:04:36
2007: poll started
Trying to connect to <ipaddressA>/110...connected.
fetchmail: POP3< +OK <21929.1171947876@svr306>
fetchmail: POP3> CAPA
fetchmail: POP3< +OK Capability list follows
fetchmail: POP3< TOP
fetchmail: POP3< UIDL
fetchmail: POP3< LAST
fetchmail: POP3< USER
fetchmail: POP3< APOP
fetchmail: POP3< STLS
fetchmail: POP3< .
fetchmail: POP3> STLS
fetchmail: POP3< +OK starting TLS negotiation
23391:error:1408F10B:SSL routines:SSL3_GET_RECORD:wrong version
number:s3_pkt.c:288:
fetchmail: <hostnameA>: opportunistic upgrade to TLS failed, trying to
continue.
fetchmail: POP3> USER <usernameA>
fetchmail: POP3< authorization first
fetchmail: Unknown login or authentication error on <usernameA>@<hostnameA>
fetchmail: POP3> QUIT
fetchmail: POP3< -ERR authorization first
fetchmail: authorization first
fetchmail: client/server protocol error while fetching from
<usernameA>@<hostnameA>
fetchmail: 6.3.7 querying <hostnameA> (protocol POP3) at Tue Feb 20 14:04:37
2007: poll completed
fetchmail: Query status=4 (PROTOCOL)
fetchmail: 6.3.7 querying <hostnameB> (protocol POP3) at Tue Feb 20 14:04:37
2007: poll started
[snip]
fetchmail: 6.3.7 querying <hostnameB> (protocol POP3) at Tue Feb 20 14:04:38
2007: poll completed
fetchmail: normal termination, status 0

$ cat .fetchmailrc
set postmaster <account>
set no bouncemail

defaults
  protocol pop3
  flush
  no mimedecode
  smtphost localhost

poll <hostnameA>
  user <usernameA>
  password <password>

poll <hostnameB>
  user <usernameB>
  password <password>
  ssl
  sslfingerprint <fingerprint>


If I put 'sslproto ""' in hostnameA, it successfully fetched.

Although this looks like a genuine server bug, it should be possible to work
around this - the only upstream concern however is that we don't go through
wastefull disconnect/reconnect cycles unless we have to. Comment #9 documents
one such case.

Thanks, Takuto!

Please reopen this bug, since comment #9 provides evidence this isn't fixed
yet.

Created an attachment (id=111758) [details]
Patch to repoll on protocol errors (such as this) as well.

This new patch (incremental to the previous for 6.3.6; self-sufficient for
6.3.7) should fix the problem reported in comment #9, by retrying the whole
poll without SSL if protocol errors occur after USER command.

the patch works fine on my env.

Thank you; I've committed the patch to the upstream SVN repository, it will
become part of fetchmail 6.3.8.

Thanks Matthias. Applied patch in 6.3.7-r1.

I don't think patching 6.3.6 (current stable) is necessary, as this is not a
common issue. If anyone disagrees, feel free to poke me and I'll bump 6.3.6
with both patches applied.