Bugzilla Bug 198914

Hi.

As openssl-0.9.8g went stable, I fail to fetch mail from one server via
fetchmail:

1247:error:14094410:SSL routines:SSL3_READ_BYTES:sslv3 alert handshake
failure:s3_pkt.c:1053:SSL alert number 40

Bear in mind that I know practically nothing about SSL/OpenSSL, but I can
verify that everything works perfectly fine with dev-libs/openssl-0.9.8f.

* * *

zealot ~ # emerge --info --ignore-default-opts
Portage 2.1.3.19 (default-linux/amd64/2007.0, gcc-4.1.2, glibc-2.6.1-r0,
2.6.23-hardened-r1 x86_64)
=================================================================
System uname: 2.6.23-hardened-r1 x86_64 AMD Athlon(tm) 64 X2 Dual Core
Processor 3800+
Timestamp of tree: Mon, 12 Nov 2007 03:46:01 +0000
app-shells/bash:     3.2_p17
dev-java/java-config: 1.3.7, 2.0.33-r1
dev-lang/python:     2.4.4-r6
dev-python/pycrypto: 2.0.1-r6
sys-apps/baselayout: 1.12.9-r2
sys-apps/sandbox:    1.2.18.1-r2
sys-devel/autoconf:  2.13, 2.61-r1
sys-devel/automake:  1.6.3, 1.7.9-r1, 1.9.6-r2, 1.10
sys-devel/binutils:  2.18-r1
sys-devel/gcc-config: 1.3.16
sys-devel/libtool:   1.5.24
virtual/os-headers:  2.6.22-r2
ACCEPT_KEYWORDS="amd64"
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-march=athlon64 -msse3 -O2 -pipe"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/kde/3.5/env /usr/kde/3.5/share/config
/usr/kde/3.5/shutdown /usr/share/X11/xkb /usr/share/config"
CONFIG_PROTECT_MASK="/etc/env.d /etc/env.d/java/ /etc/fonts/fonts.conf
/etc/gconf /etc/revdep-rebuild /etc/terminfo /etc/texmf/web2c /etc/udev/rules.d
/usr/kde/*/share/apps/kjava/kjava.policy /usr/kde/*/shutdown/agent-shutdown.sh
/usr/kde/3.5/share/services/rlogin.protocol"
CXXFLAGS="-march=athlon64 -msse3 -O2 -pipe"
DISTDIR="/usr/portage/distfiles"
EMERGE_DEFAULT_OPTS="--ask --verbose"
FEATURES="buildpkg collision-protect distlocks fixpackages metadata-transfer
multilib-strict parallel-fetch sandbox sfperms suidctl unmerge-orphans
userfetch userpviv usersandbox"
GENTOO_MIRRORS="http://trumpetti.atm.tut.fi/gentoo
http://mirror.uni-c.dk/pub/gentoo http://ds.thn.htu.se/linux/gentoo
http://mirror.gentoo.no/ http://distfiles.gentoo.org"
LANG="en_US.utf8"
LC_ALL="en_US.utf8"
LINGUAS="en en_GB en_US"
MAKEOPTS="-j3"
PKGDIR="/usr/portage/packages"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress
--force --whole-file --delete --delete-after --stats --timeout=180
--exclude=/distfiles --exclude=/local --exclude=/packages
--filter=H_**/files/digest-*"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
PORTDIR_OVERLAY="/usr/local/portage"
SYNC="rsync://rsync.europe.gentoo.org/gentoo-portage"
USE="3dnow 3dnowext X alsa amd64 berkdb bitmap-fonts bzip2 cli cracklib crypt
cups curl dri fortran gdbm gif gpm iconv isdnlog jpeg kde kdehiddenvisibility
midi mmx mmxext mudflap ncurses nls nptl nptlonly opengl openmp pam pcre perl
png pppd python qt3 readline reflection session spell spl sse sse2 ssl tcl
tcltk tcpd threads tiff tk truetype truetype-fonts type1-fonts unicode
xcomposite xinerama xorg zlib" ALSA_CARDS="emu10k1 intel8x0"
ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file
hooks iec958 ioplug ladspa lfloat linear meter mulaw multi null plug rate route
share shm softvol" ELIBC="glibc" INPUT_DEVICES="keyboard mouse" KERNEL="linux"
LINGUAS="en en_GB en_US" USERLAND="GNU" VIDEO_CARDS="nv"
Unset:  CPPFLAGS, CTARGET, INSTALL_MASK, LDFLAGS, PORTAGE_COMPRESS,
PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS

I forgot to mention: all tests are passed with openssl-0.9.8g.

Hi,

I have a similar problem with net-irc/xchat-2.8.4 and dev-libs/openssl-0.9.8g:

[08:46:06] --- Looking up irc.mozilla.org..
[08:46:06] --- Connecting to irc.mozilla.org (63.245.208.159) port 6697..
[08:46:07] --- Connection failed. Error: (336151568) error:14094410:SSL
routines:SSL3_READ_BYTES:sslv3 alert handshake failure

This doesn't happen on all networks/servers but for irc.mozilla.org it's 100%
reproduceable.
I don't know if this problem started after upgrade to openssl-0.9.8g or after
upgrade to glibc-2.6.1...

Cheers
Poly-C

Created an attachment (id=136861) [details]
emerge --info

P.S.: I know, I'm using masked app-shells/bash-3.2_p25 but that's intentional
(I already patched portage to work with this bash version) and this problem
already occured with bash-3.2_p17...

Confirming openssl-0.9.8g causing xchat amongst others to fail SSL handshakes.

emerge --info:
Portage 2.1.4_rc3 (default-linux/x86/2006.1, gcc-4.1.2, glibc-2.7-r0,
2.6.22-gentoo-r9 i686)
=================================================================
System uname: 2.6.22-gentoo-r9 i686 AMD Athlon(tm) XP 2400+
Timestamp of tree: Tue, 27 Nov 2007 23:00:01 +0000
ccache version 2.4 [enabled]
app-shells/bash:     3.2_p17
dev-java/java-config: 1.3.7, 2.1.2-r1
dev-lang/python:     2.4.4-r4, 2.5.1-r4
dev-python/pycrypto: 2.0.1-r6
dev-util/ccache:     2.4-r7
dev-util/confcache:  0.4.2-r1
sys-apps/baselayout: 1.12.10-r5
sys-apps/sandbox:    1.2.18.1-r2
sys-devel/autoconf:  2.13, 2.61-r1
sys-devel/automake:  1.4_p6, 1.5, 1.6.3, 1.7.9-r1, 1.8.5-r3, 1.9.6-r2, 1.10
sys-devel/binutils:  2.18-r1
sys-devel/gcc-config: 1.3.16
sys-devel/libtool:   1.5.24
virtual/os-headers:  2.6.22-r2
ACCEPT_KEYWORDS="x86"
CBUILD="i686-pc-linux-gnu"
CFLAGS="-O2 -mtune=athlon -fomit-frame-pointer -march=athlon-xp -pipe"
CHOST="i686-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/kde/3.3/env /usr/kde/3.3/share/config
/usr/kde/3.3/shutdown /usr/kde/3.5/env /usr/kde/3.5/share/config
/usr/kde/3.5/shutdown /usr/share/config"
CONFIG_PROTECT_MASK="/etc/env.d /etc/env.d/java/ /etc/fonts/fonts.conf
/etc/gconf /etc/revdep-rebuild /etc/terminfo /etc/udev/rules.d"
CXXFLAGS="-O2 -mtune=athlon -fomit-frame-pointer -march=athlon-xp -pipe"
DISTDIR="/usr/portage/distfiles"
EMERGE_DEFAULT_OPTS="--with-bdeps y"
FEATURES="ccache distlocks metadata-transfer parallel-fetch sandbox sfperms
strict unmerge-orphans userfetch"
GENTOO_MIRRORS="http://mirror.gentoo.no/ ftp://ftp.du.se/pub/os/gentoo
http://ds.thn.htu.se/linux/gentoo http://gd.tuwien.ac.at/opsys/linux/gentoo/"
MAKEOPTS="-j2"
PKGDIR="/usr/portage/packages"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress
--force --whole-file --delete --delete-after --stats --timeout=180
--exclude=/distfiles --exclude=/local --exclude=/packages
--filter=H_**/files/digest-*"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
PORTDIR_OVERLAY="/usr/local/portage"
SYNC="rsync://rsync.gentoo.org/gentoo-portage"
USE="3dnow 3dnowext X aiglx async bcmath beagle berkdb bitmap-fonts bittorrent
bootsplash bzip2 cairo ccache chroot clamav cli contentcache cpudetection
cracklib crypt css cups curl dbus dri dvd emerald epiphany fbsplash ffmpeg
finger firefox flac font-server fontconfig fortran ftp gaim gd gdbm glitz gmail
gmailtimestamps gnome gnome-print gnutls gpm gstreamer gtalk gtk gtk2 hal iconv
icq imap ipv6 isdnlog jabber java javascript jpeg kde lm_sensors logitech-mouse
maildir matroska mbox midi mime mmap mmx mono mp3 mp4 mpeg mplayer mudflap
ncurses nls nptl nptlonly nsplugin ogg opengl openmp pam pam_chroot
pam_timestamp pango pcre pdf perl png ppds python qt3 qt4 readline reflection
reiser4 reiserfs ruby samba session snmp spell spl sqlite3 sse sse-filters ssl
stream subversion svg symlink tcpd theora threads tiff tk truetype-fonts
type1-fonts unicode vcd vorbis wv x264 x86 xface xorg xv yahoo zlib"
ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1
emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m
maestro3 trident usb-audio via82xx via82xx-modem ymfpci"
ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file
hooks iec958 ioplug ladspa lfloat linear meter mulaw multi null plug rate route
share shm softvol" ELIBC="glibc" INPUT_DEVICES="keyboard mouse evdev"
KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001
mtxorb ncurses text" USERLAND="GNU" VIDEO_CARDS="nvidia vesa"
Unset:  CPPFLAGS, CTARGET, INSTALL_MASK, LANG, LC_ALL, LDFLAGS, LINGUAS,
PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS

Not sure if it's exactly the same problem, but I cannot connect to google talk
with net-im/sim and openssl-0.9.8g, and 0.9.8f works fine
There's a bug report that suggests that on debian, both versions work (so maybe
they have some patch?):
http://developer.berlios.de/bugs/?func=detailbug&bug_id=12510&group_id=4482

just as an FYI, this is in OpenSSL's request tracker with a patch an
explanation:
http://rt.openssl.org/index.html?q=1629

we added enable-tlsext by default in 0.9.8g ... if you remove that from the
ebuild, things should probably work

Right, disabling tlsext helped with my issue from comment 5. Making it a USE
flag would make it easier, but your call, dunno how far is a fixed upstream
version.

a USE flag would just ignore the issue ... and then people would just complain
about the same thing when the flag was turned on ...

has anyone tried the upstream patch (user/pass = guest/guest) ?  we can either
apply that or just drop tlsext until it's fixed ... it was only added on a lark
anyways; nothing is needing it at the moment

(In reply to comment #9)
> a USE flag would just ignore the issue ... and then people would just complain
> about the same thing when the flag was turned on ...
> 
> has anyone tried the upstream patch (user/pass = guest/guest) ?

Not yet, I guessed it's not upstream patch but an user attached patch (please
prove me wrong) and that no upstream dev commented on it yet.
(is your guest/guest also from bugmenot or something that's suggested on the
page itself? couldn't even find a "register" link, not to mention the tracker
is localised to czech in a fugly way and there's no english option :)

> we can either
> apply that or just drop tlsext until it's fixed ... it was only added on a lark
> anyways; nothing is needing it at the moment

I can test and it will probably work, just questioning the upstream intention.

I have similar results: openssl-0.9.8g doesn't work on one particular e-mail
server, but 0.9.8f works fine.

If someone cares to supply an ebuild with the potential patch, I'll happily
test it and report whether the patch fixes it. But I would urge either patching
this or removing the offending flag, especially seeing as it doesn't appear to
be needed by anything.

the guest/guest part i read somewhere in the openssl docs ... dont ask me
where; their site is a pita to find stuff :p

will people stop posting that "downgrading to 0.9.8f fixes things" ... we dont
care about the version difference.  all that matters is removing of the tlsext
flag in the 0.9.8g ebuild and things work after that.

I feel the proper resolution here is going to be to merge the patch provided in
OpenSSL's RT. Some servers can not handle TLS (which is technically SSLv3.1)
Extensions (which are present in TLSv1.1/SSLv3.2) on SSLv3 connections. It's a
simple version check and omission of the extensions.

If we've got SSLv3.0, don't offer extensions since extension support wasn't
formally in a spec until SSLv3.1. If we have at least SSLv3.1, offer
extensions. It seems straight forward.

Additionally, based on Caster's bug link I reduced this down to a simple test
case.

$ openssl s_client -ssl3 -no_ssl2 -showcerts -connect talk.google.com:5223

In doing so I tested the supplied patch in OpenSSL's RT. This also resolves
several complains people have had with Pidgin and Google Talk.

Please use openssl-0.9.8g-r1

(In reply to comment #13)
> $ openssl s_client -ssl3 -no_ssl2 -showcerts -connect talk.google.com:5223

I can confirm this test case now works in -r1, however:

> In doing so I tested the supplied patch in OpenSSL's RT. This also resolves
> several complains people have had with Pidgin and Google Talk.

Unfortunately SIM still doesn't work with Google Talk (but works without
enable-tlsext in -r0). Either the patch is incomplete and the extensions are
also used elsewhere, or SIM sets the connection in a way that doesn't trigger
the if clauses? Unfortunately can't see from the intergrated network monitor if
the error seems different than without patch.

What port is SIM connecting to over at Google Talk? 5222 is the default Google
Talk port which uses TLS. Where as 5223 appears to use SSLv3.

This patch allowed be to be successful in connecting to Google Talk via both
ports using Pidgin, while not using the patch only allowed me to connect via
5222.

(In reply to comment #15)
> What port is SIM connecting to over at Google Talk? 5222 is the default Google
> Talk port which uses TLS. Where as 5223 appears to use SSLv3.

SIM uses 5223, I think it never worked with 5222, including now.

(In reply to comment #14)

Thanks. The same sim problems and fix here.
The -r1 patch doesn't work fully.

Still need code to re-produce or steps to reproduce this on my machine OTHER
then installing/using sim.

I've had a look at the SIM code and it looks like it is using TLSv1_method in
sim/sslclient.cpp to connect.  Patching it to use SSLv23_client_method allows
it to work with the -r1 patch.

This can be tested with:

$ openssl s_client -tls1 -no_ssl2 -showcerts -connect talk.google.com:5223

which works without enable-tlsext but fails with enable-tlsext enabled even
with  the -r1 patch.

This seems to be critical as openssl-0.9.8g has been marked as stable since
November and is also installed in stage3-i686-2008.0_beta1.

Created an attachment (id=150701) [details]
sim-0.9.4.3-sslv23.patch

Thanks Matthew, nice one. Everything is working fine now. I guess I'll need to
file a separate bug report for it. And openssl -r1 should really go stable now.

(In reply to comment #21)
> Thanks Matthew, nice one. Everything is working fine now. I guess I'll need to
> file a separate bug report for it. And openssl -r1 should really go stable now.

Um, the question is if sim is really doing something it shouldn't, and tlsext
just revealed it, or if it's still a problem of openssl (or perhaps the gmail
server) and the patch is just working around it?

(In reply to comment #22)
> (In reply to comment #21)
> > Thanks Matthew, nice one. Everything is working fine now. I guess I'll need to
> > file a separate bug report for it. And openssl -r1 should really go stable now.
> 
> Um, the question is if sim is really doing something it shouldn't, and tlsext
> just revealed it, or if it's still a problem of openssl (or perhaps the gmail
> server) and the patch is just working around it?
> 

SIM is connecting to a known SSL-only port with TLSv1 and relying on the
fallback to SSLv3. I'd say it's a broken assumption on SIM's part simply
because if you advertised "Connect to my SSL port" and you ran SSLv2
(stupidly), then TLSv1_client_method would not work. As I previously explained,
TLSv1 is really SSLv3.1, which is why TLSv1_client_method works when connecting
to an SSLv3 server.... usually. However, TLS has a few things in it's spec that
SSLv3 stuff doesn't expect and SSLv3 implementations don't have to follow the
"if you see something you don't know, ignore it" that TLS follows (due to
extensions).

I know at least 1 more https server which require openssl -r1 so it's not
gmail's problem and the openssl patch looks good.

The patch for sim is more a workaround for now. I think they should redesign
ssl init sections and do something similar what stunnel does,
stunnel-4.21/src/options.c:
    /* sslVersion */
    switch(cmd) {
    case CMD_INIT:
        section->client_method=SSLv3_client_method;
        break;
    case CMD_EXEC:
        if(strcasecmp(opt, "sslVersion"))
            break;
        if(!strcasecmp(arg, "all")) {
            section->client_method=SSLv23_client_method;
        } else if(!strcasecmp(arg, "SSLv2")) {
            section->client_method=SSLv2_client_method;
        } else if(!strcasecmp(arg, "SSLv3")) {
            section->client_method=SSLv3_client_method;
        } else if(!strcasecmp(arg, "TLSv1")) {
            section->client_method=TLSv1_client_method;
        } else
            return "Incorrect version of SSL protocol";

but I'm not an expert here, so it's just IMHO.

I'm apparently hitting the very same or a similar problem when trying to
connect to cacert.org:

colo@zealot ~ $ openssl s_client -connect cacert.org:443 -no_ssl2 -bugs
CONNECTED(00000003)
[... snip ...]
verify return:1
1660:error:140943FC:SSL routines:SSL3_READ_BYTES:sslv3 alert bad record
mac:s3_pkt.c:1053:SSL alert number 20
1660:error:140790E5:SSL routines:SSL23_WRITE:ssl handshake
failure:s23_lib.c:188:
colo@zealot ~ $ openssl version
OpenSSL 0.9.8g 19 Oct 2007


~amd64:
* dev-libs/openssl [R 0.9.8g-r1] <target>
    Reasons: app-misc/ca-certificates-20070303-r1:0::installed
    -bindist -gmp -kerberos (sse2) -test zlib

CHOST="x86_64-pc-linux-gnu"
CFLAGS="-O2 -pipe -march=nocona -fomit-frame-pointer"

(In reply to comment #25)
> I'm apparently hitting the very same or a similar problem when trying to
> connect to cacert.org:

I believe you're having a different error since I'm able to reproduce this on
multiple versions of OpenSSL. 0.9.8e, 0.9.8f, 0.9.8g

(In reply to comment #25)
> I'm apparently hitting the very same or a similar problem when trying to
> connect to cacert.org:
> 
> colo@zealot ~ $ openssl s_client -connect cacert.org:443 -no_ssl2 -bugs
> CONNECTED(00000003)
> [... snip ...]
> verify return:1
> 1660:error:140943FC:SSL routines:SSL3_READ_BYTES:sslv3 alert bad record
> mac:s3_pkt.c:1053:SSL alert number 20
> 1660:error:140790E5:SSL routines:SSL23_WRITE:ssl handshake
> failure:s23_lib.c:188:

However, additionally, you're connecting to an SSLv2/SSLv3 port with
TLSv1/TLSv1.1, this is an improper way to connect and you should fix the code
that spawned you to write this test case. 

BTW the patch for sim was applied in svn and works fine here. But it's
available only in sim-9999 live svn, so people can still hit the problem. I
guess a separate bug for sim should be opened?
As for openssl itself, looks like upstream didn't apply the patch in 0.9.8h and
we apply it ourselves.

I already suggested to split it in the comment #21, so here we go, bug #224095
You might want to close this bug now and/or investigate why the patch for
openssl hasn't been included into the version 0.9.8h.

(In reply to comment #29)
> I already suggested to split it in the comment #21, so here we go, bug #224095
> You might want to close this bug now and/or investigate why the patch for
> openssl hasn't been included into the version 0.9.8h.
> 

Because 0.9.8h wasn't a bug fix release, it was a security fix release.