Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 90694 - qmail-smtpd randomly(?) segfaults, hardened gentoo x86_64
Summary: qmail-smtpd randomly(?) segfaults, hardened gentoo x86_64
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: [OLD] Server (show other bugs)
Hardware: AMD64 Linux
: High normal (vote)
Assignee: The Gentoo Linux Hardened Team
Depends on:
Reported: 2005-04-28 02:44 UTC by Luca Lesinigo
Modified: 2005-08-18 15:58 UTC (History)
1 user (show)

See Also:
Package list:
Runtime testing required: ---


Note You need to log in before you can comment on or make changes to this bug.
Description Luca Lesinigo 2005-04-28 02:44:36 UTC
I've got an x86_64 server with hardened gentoo (see below for details), using qmail as MTA (see below for details :), and qmail-smtpd sometimes, not always, segfaults.

From the logs I see:
Apr 28 11:13:48 fit qmail-smtpd[32355]: segfault at 00000000701b3270 rip 0000003652616904 rsp 00000076b94f6450 error 4
(note: segfault addresses always ends with 0, rip always ends with 904, rsp always starts with 0000007 and always ends with 0)
Apr 28 11:13:48 fit grsec: From W.X.Y.Z: signal 11 sent to /var/mail/qmail/bin/qmail-smtpd[qmail-smtpd:32355] uid/euid:201/201 gid/egid:200/200, parent /usr/bin/tcpserver[tcpserver:694] uid/euid:201/201 gid/egid:200/200

Unfortunately, I had no luck in finding why it segfaults and when. This is a production mail server, and when the traffic was really low I never noticied the problem. I found it now because some mails started taking hours to arrive, and usually a bunch of them arrives all together...
I put a cron job on a remote server sending me a mail with a timestamp every 10 minutes: many of them arrive in just some seconds. Then they stop coming, maybe for an hour, and then the delayed mails arrive all at the same time, when the remote SMTP server tries to reconnect to this, faulty, one.

I don't know if this problem is x86_64-specific, or hardened-specific, or whatever. This is the only x86_64 hardware I own now, and it's a remote server, so I cannot test different kernels on this machine (I've got no serial console to it) or test the same setup on other similar, x86_64, machines.
I'll have a spare computer next week, I'll try to reproduce the setup there but it's gonna be an i386, not an x86_64.
Also I will try a vanilla kernel if I happen to get where the server is colocated.

I really have no other ideas where to look...

Reproducible: Always
Steps to Reproduce:
1. Get a mail domain on my server ;)
2. Send mail to an account on that domain!

Actual Results:  
Sometimes qmail-smtpd segfaults, so remote SMTP servers delay mail delivery to a later time.
Sometimes it works ok, at least mail arrives sooner or later...

Expected Results:  
qmail-smtpd shouldn't segfault.

Here are all the gory details...

- Dell SC1425, dual Xeon EM64T HTT with ECC ram and sata disks (no error from hardware)
- using linux raid1 software on two disks
- note: hardware is new and has no other problems. Normal load is near "nothing" (currently it is 
serving mail and web for two really low-traffic domains, and nothing else), but the server is stable even 
under high load (for testing purposes)

Operating system:
- Hardened Gentoo x86_64
- grsec and PaX active
- pie & ssp active
- kernel 2.6.10-hardened-r3, SMP
- gcc (GCC) 3.4.3 20041125 (Gentoo Hardened Linux 3.4.3-r1, ssp-3.4.3-0, pie-8.7.7)

- mail-mta/qmail-1.03-r15  +noauthcram +notlsbeforeauth (-selinux) +ssl
- mail-filter/qmail-scanner-1.25-r1  -spamassassin
- net-mail/relay-ctrl-3.1.1-r2
- net-mail/vpopmail-5.4.6-r1  +clearpasswd -ipalias -mysql

emerge info - please note I also tried recompiling qmail with CFLAGS=CXXFLAGS=MAKEOPTS="", and 
also turning off ssp:

Portage (hardened/amd64, gcc-3.4.3, glibc-, 2.6.10-hardened-r3-prod 
System uname: 2.6.10-hardened-r3-prod x86_64 Intel(R) Xeon(TM) CPU 2.80GHz
Gentoo Base System version 1.4.16
Python:              dev-lang/python-2.3.4-r1 [2.3.4 (#1, Feb 15 2005, 17:17:06)]
dev-lang/python:     2.3.4-r1
sys-devel/autoconf:  2.13, 2.59-r6
sys-devel/automake:  1.5, 1.8.5-r3, 1.6.3, 1.7.9-r1, 1.4_p6, 1.9.4
sys-devel/libtool:   1.5.14
CFLAGS="-O -march=nocona -pipe"
CONFIG_PROTECT="/etc /usr/kde/2/share/config /usr/kde/3/share/config /usr/share/config /var/
qmail/alias /var/qmail/control /var/vpopmail/domains /var/vpopmail/etc"
CONFIG_PROTECT_MASK="/etc/gconf /etc/terminfo /etc/env.d"
CXXFLAGS="-O -march=nocona -pipe"
FEATURES="autoaddcvs autoconfig ccache distlocks sandbox strict"
USE="amd64 apache2 bash-completion berkdb clearpasswd crypt curl gd gdbm hardened imap innodb 
ipv6 javascript jpeg justify libwww mysql ncurses nls noauthcram notlsbeforeauth pam perl pic pie png 
python readline semanticfix snmp ssl tcpd tiff userlocales vhosts xml2 zlib"
Comment 1 solar (RETIRED) gentoo-dev 2005-04-28 05:36:19 UTC
Lowering Severity. amd64 hardened is not even officially supported. 
qmail-smtpd works fine under hardened envionments.
Comment 2 Robin Johnson archtester Gentoo Infrastructure gentoo-dev Security 2005-04-28 12:53:01 UTC
enable core dumps and recompile qmail with debugging.
then wait for one of these emails to come in, and generate a backtrace using the core dump.

CFLAGS="-O -march=nocona -pipe -ggdb3"
Comment 3 Michael Hanselmann (hansmi) (RETIRED) gentoo-dev 2005-06-05 07:44:44 UTC
Can you provide the information, please?
Comment 4 Michael Hanselmann (hansmi) (RETIRED) gentoo-dev 2005-08-18 15:58:15 UTC
No response.