Bug 457654 – net-irc/inspircd-2.0.10 - error: Handshake Failed - The Diffie-Hellman prime sent by the server is not acceptable (not long enough).

• Product: Gentoo Linux
• Status: RESOLVED FIXED

[Description michael 2013-02-15 06:01:45 UTC]

*note* going to combine a few issues the devs upstream noted as well with the ebuild.

Unable to link two or more inspircd servers. Anope, Clients, and Denora all connect fine to servers. the only ones that use ssl are inspircd servers and clients 
all effected servers are using gentoo ebuilds, standard tarball installs noted to work fine upstream.
at the time i filed bug upstream I was running the following 

[I] net-irc/inspircd
Installed versions: 2.0.9(23:44:29 02/01/13)(geoip gnutls ipv6 mysql ssl -ldap -postgres -sqlite)

*this has been updated to 2.0.10 and problem persists*

[I] net-libs/gnutls
Installed versions: 2.12.20(07:57:13 01/31/13)(cxx nettle nls zlib -bindist -doc -examples -guile -lzo -pkcs11 -static-libs -test)

debug log on leaf:
Mon Feb 4 08:43:27 2013: New file descriptor: 16
Mon Feb 4 08:43:27 2013: BufferedSocket::DoConnect success
Mon Feb 4 08:43:27 2013: LINK: Connection to ^Bamericas.ircforex.com^B[184.106.81.186] started.
Mon Feb 4 08:43:27 2013: Error on FD 16 - 'Handshake Failed - The Diffie-Hellman prime sent by the server is not acceptable (not long enough).'
Mon Feb 4 08:43:27 2013: LINK: Connection to '^Bamericas.ircforex.com^B' failed with error: Handshake Failed - The Diffie-Hellman prime sent by the server is not acceptable (not long enough).
Mon Feb 4 08:43:30 2013: DoWrite on errored or closed socket
Mon Feb 4 08:43:30 2013: Remove file descriptor: 16
Mon Feb 4 08:43:30 2013: LINK: Connection to '^Bamericas.ircforex.com^B' failed.
Mon Feb 4 08:43:30 2013: LINK: Connection to '^Bamericas.ircforex.com^B' was established for 3s
Mon Feb 4 08:43:31 2013: Deleting 10TreeSocket @0xadf0f0

debug on hub:
Mon Feb 4 08:47:28 2013: HandleEvent for Listensocket 184.106.81.186:7778 nfd=88
Mon Feb 4 08:47:28 2013: classbase::+ @0x7f87d802b010
Mon Feb 4 08:47:28 2013: New file descriptor: 88
Mon Feb 4 08:47:31 2013: Error on FD 88 - 'Handshake Failed - A TLS packet with unexpected length was received.'
Mon Feb 4 08:47:31 2013: LINK: Connection to '^Binbound from 185.14.185.167^B' failed with error: Handshake Failed - A TLS packet with unexpected length was received.
Mon Feb 4 08:47:35 2013: DoWrite on errored or closed socket
Mon Feb 4 08:47:35 2013: Remove file descriptor: 88
Mon Feb 4 08:47:35 2013: LINK: Connection to '^Binbound from 185.14.185.167^B' failed.
Mon Feb 4 08:47:35 2013: LINK: Connection to '^Binbound from 185.14.185.167^B' was established for 7s
Mon Feb 4 08:47:35 2013: Deleting 10TreeSocket @0x7f87d802b010

some notes by the devs upstream:
ChrisTX~
Btw, it doesn't seem to take care of any regex modules, so neither the PCRE and TRE regex modules are offered by use flags and the POSIX regex module isn't ever being built. If you need any of these (most people do in some way) you'll have to edit the ebuild.

Btw2, the ebuild authors should have made essl and the SSL USE flag do the same thing and just append --enable-gnutls/openssl instead of the one copying it from extras and the other adding this configure flag.

*note* openssl is nonfunctional in inspircd. even though gnutls is recommended openssl is still supported but not enabled by the use flags in the ebuild.

[Comment 1 Chema Alonso Josa (RETIRED) 2013-02-18 11:56:25 UTC]

I'll take a look a this shortly.

Thanks for reporting.

[Comment 2 Chema Alonso Josa (RETIRED) 2013-02-19 23:08:26 UTC]

1. I haven't enough time to reproduce the handshake failure, but I've done some research. The error is raised inside the gnutls library (auth_dh_common.c). I'll try to reproduce it when I have more time.
Could you try raising the value of the dhbits parameter to one of: 2048, 3072 or 4096 to see if the problem persists?

2. I've bumped the revision to inspircd-2.0.10-r1 to include the flags suggested by upstream and tunned the configuration phase. Following the indications in the wiki (http://wiki.inspircd.org/Modules/2.0/ssl_openssl) I've setup openssl with no problem. Could you test the new revision to check if that helped?

3. To get rid of these messages:
Mon Feb 4 18:27:51 2013: m_ssl_gnutls.so: Failed to set X.509 trust file '/etc/inspircd/ca.pem': Error while reading file.
Mon Feb 4 18:27:51 2013: m_ssl_gnutls.so: Failed to set X.509 CRL file '/etc/inspircd/crl.pem': Error while reading file.
You need to include the proper Certificate Authority and Certificate Revocation List files.

I think that covers all issues reported in the bug. If I dropped any other issue, please let me know.

Thx.

[Comment 3 michael 2013-02-21 03:45:18 UTC]

I have manipulated the dh-bits flag as suggested upstream and it had no effect whatsoever. I am currently updating to r1 and will give an update later, most likely Friday evening since it will affect a live server, Friday evenings are actually a slow day.

here is the bug that was posted upstream for reference. https://github.com/inspircd/inspircd/issues/421

[Comment 4 michael 2013-02-23 01:31:48 UTC]

Oddly r1 seems to have resolved the gnutls issue as well. just linked up without having to use openssl. I'll close the bug.

[Comment 5 Chema Alonso Josa (RETIRED) 2013-02-25 10:07:38 UTC]

Good to hear that. Probably the use of econf messed things up. The use of ./configure as in version =net-irc/inspircd-2.0.9 seems to fix the problem.

Cheers.