I upgraded qmail from -r15 to -r16. I use a smarthost which i relay my mail to with smtp-auth. With -r15 I had my username and password in smtproutes base64 encoded but the smarthost complained about bad authentification and my base64 encoded username showed up in the logs (vlog mysql-table from vpopmail). So I changed the entry in smtproutes to plaintext. The vpopmail-logs on the smarthost show the correct username and password and that the authentification was successful, but my qmail-remote crashes: @4000000042f48b8125ea61fc starting delivery 3: msg 950098 to remote email@address.com @4000000042f48b8125ea7d54 status: local 0/10 remote 1/20 @4000000042f48b821d5c0f54 delivery 3: deferral: qmail-remote_crashed./ I also tried to strace qmail-remote, but I don't understand much of it. I'll attach a file with the strace-output. It seems I'm not the only one, whose qmail-remote crashes, but the strace-output looks different, I think. Reproducible: Always Steps to Reproduce: 1. 2. 3. Portage 2.0.51.22-r2 (default-linux/x86/2005.0, gcc-3.4.4, glibc-2.3.5-r1, 2.6.12-gentoo-r6 i686) ================================================================= System uname: 2.6.12-gentoo-r6 i686 AMD Athlon(tm) processor Gentoo Base System version 1.6.13 dev-lang/python: 2.3.5, 2.4.1-r1 sys-apps/sandbox: 1.2.11 sys-devel/autoconf: 2.13, 2.59-r7 sys-devel/automake: 1.4_p6, 1.5, 1.6.3, 1.7.9-r1, 1.8.5-r3, 1.9.6 sys-devel/binutils: 2.16.1 sys-devel/libtool: 1.5.18-r1 virtual/os-headers: 2.6.11-r2 ACCEPT_KEYWORDS="x86 ~x86" AUTOCLEAN="yes" CBUILD="i686-pc-linux-gnu" CFLAGS="-mtune=athlon-tbird -O2 -fomit-frame-pointer -pipe" CHOST="i686-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/kde/2/share/config /usr/kde/3/share/config /usr/lib/X11/xkb /usr/lib/fax /usr/share/config /var/qmail/alias /var/qmail/control /var/spool/fax/etc" CONFIG_PROTECT_MASK="/etc/gconf /etc/terminfo /etc/env.d" CXXFLAGS="-mtune=athlon-tbird -O2 -fomit-frame-pointer -pipe" DISTDIR="/usr/portage/distfiles" FEATURES="autoconfig distlocks sandbox sfperms strict" GENTOO_MIRRORS="http://ftp.belnet.be/mirror/rsync.gentoo.org/gentoo/ ftp://ftp.belnet.be/mirror/rsync.gentoo.org/gentoo/ http://pandemonium.tiscali.de/pub/gentoo/ http://194.117.143.71" LANG="german" LC_ALL="de_DE.utf8" MAKEOPTS="-j2" PKGDIR="/usr/portage/packages" PORTAGE_TMPDIR="/var/tmp" PORTDIR="/usr/portage" PORTDIR_OVERLAY="/usr/local/portage" SYNC="rsync://rsync.europe.gentoo.org/gentoo-portage" USE="x86 3dnow aalib acl acpi apache2 apm avi bash-completion berkdb bzlib cairo crypt cups curl dba dbx dedicated dhcp divx4linux dvd dvdread dynagraph encode ethereal fam fastcgi foomaticdb ftp gd gif glitz graphviz icq imagemagick imap imlib innodb ipv6 jpeg libcaca libclamav libwww lm_sensors mad maildir mailwrapper mime mmx mpeg mpm-worker mssql mysql mythtv ncurses network nls nocd nptl oav odbc oggvorbis opengl pam pcre pear-db perl php png pnp postgres ppds prelude python qmail quicktime readline real rrdtool samba session slang snmp soap sockets spamassassin spell spl ssl svg tcpd threads tidy tiff tokenizer truetype underscores unicode usb v4l v4l2 vhosts winbind wmf xml xml2 xsl xslt xv xvid zlib fritzcapi_cards_fcpci userland_GNU kernel_linux elibc_glibc" Unset: ASFLAGS, CTARGET, LDFLAGS, LINGUAS
Created attachment 65222 [details] strace output of crashing qmail-remote
Have you recently remerged qmail-1.03-r16? I fixed a bug that caused qmail-remote to crash about two weeks ago.
Yes, I emerged qmail-1.0.3-r16 yesterday and re-emerged today after a fresh portage sync.
Can you please submit debug information as described in bug 99497?
I will, but how do I "let qmail-remote run, so it generates a core dump"? When it crashes, it doesn't generate one by itself.
You have to enable core dumps. See, for example, here: http://www.unixguide.net/linux/faq/07.14.shtml
Ok, so this is the output of gdb (it doesn't look *that* helpful for me ;)): (gdb) bt #0 0x0804fa64 in ?? () #1 0xbfeb8ed8 in ?? () #2 0x0804fa18 in ?? () #3 0x00000000 in ?? () #4 0x080504d2 in _IO_stdin_used () #5 0x00000001 in ?? () #6 0x08049b6e in ?? () #7 0xbfeb8edb in ?? () #8 0x000000fa in ?? () #9 0xbfeb8ee8 in ?? () #10 0x080495dd in ?? () #11 0x08051324 in ?? () #12 0x00000000 in ?? () #13 0xbfeb8f08 in ?? () #14 0x0804ad54 in ?? () #15 0x00000000 in ?? () #16 0x0805043d in _IO_stdin_used () #17 0x0000000f in ?? () #18 0x00000000 in ?? () #19 0x00000000 in ?? () #20 0x08051d00 in ?? () #21 0xbfeb8f58 in ?? () #22 0x0804bbf9 in ?? () #23 0x08051d00 in ?? () #24 0x00000000 in ?? () #25 0x08051e90 in ?? () #26 0x00000019 in ?? () #27 0x0000003c in ?? () #28 0x08051108 in ?? () #29 0xbfeb8f38 in ?? () #30 0x08049005 in _init () Previous frame inner to this frame (corrupt stack?) (gdb)
Please recompile qmail with these CFLAGS before: CFLAGS="-g -pipe"
I'm not sure, if I'm doing something wrong, but there's not much more information this time. I ran the following command: CFLAGS="-g -pipe" emerge -v qmail the first gdb output was from qmail compiled like this: CFLAGS="-ggdb3 -O1" emerge -v qmail (gdb) bt #0 0x08050db7 in ?? () #1 0x00000000 in ?? () #2 0x00000000 in ?? () #3 0xbf933ae8 in ?? () #4 0x08050d62 in ?? () #5 0x00000000 in ?? () #6 0x08052297 in _IO_stdin_used () #7 0x00000001 in ?? () #8 0x08049c2a in ?? () #9 0x42f52d75 in ?? () #10 0x00000000 in ?? () #11 0xbf933af8 in ?? () #12 0x0804964f in ?? () #13 0x08053784 in ?? () #14 0x00000000 in ?? () #15 0xbf933b28 in ?? () #16 0x0804b02c in ?? () #17 0x00000000 in ?? () #18 0x08052131 in _IO_stdin_used () #19 0x0000000f in ?? () #20 0x00000000 in ?? () #21 0x00000000 in ?? () #22 0x08053dd4 in ?? () #23 0x08055e80 in ?? () #24 0x00000000 in ?? () #25 0x00000000 in ?? () #26 0x000000fa in ?? () #27 0xbf933b88 in ?? () #28 0x0804c105 in ?? () #29 0x08054160 in ?? () #30 0x00000000 in ?? () #31 0x080542f0 in ?? () #32 0x00000000 in ?? () #33 0x0000003c in ?? () #34 0x00000000 in ?? () #35 0xbf935716 in ?? () #36 0xb7cca2d1 in __guard_setup () from /lib/tls/libc.so.6 Previous frame inner to this frame (corrupt stack?) (gdb)
Oh, I'm sorry, thanks for the info. I'll have a look at this somewhen this week. Looks like a problem with glibc, openssl or the qmail code.
Can you please recompile your glibc and openssl without -fomit-frame-pointers? The best would be if you could use CFLAGS="-ggdb3 -O0". That way we should have the function names in the backtrace.
unfortunatly my glibc failes to compile with these settings: i686-pc-linux-gnu-gcc iovsprintf.c -c -std=gnu99 -O0 -O2 -Wall -Winline -Wstrict-prototypes -Wwrite-strings -ggdb3 -mpreferred-stack-boundary=2 -fPIC -I../include -I. -I/var/tmp/portage/glibc-2.3.5-r1/work/build-default-i686-pc-linux-gnu-linuxthreads/libio -I.. -I../libio -I/var/tmp/portage/glibc-2.3.5-r1/work/build-default-i686-pc-linux-gnu-linuxthreads -I../sysdeps/i386/elf -I../linuxthreads/sysdeps/unix/sysv/linux/i386 -I../linuxthreads/sysdeps/unix/sysv/linux -I../linuxthreads/sysdeps/pthread -I../sysdeps/pthread -I../linuxthreads/sysdeps/unix/sysv -I../linuxthreads/sysdeps/unix -I../linuxthreads/sysdeps/i386/i686 -I../linuxthreads/sysdeps/i386 -I../libidn/sysdeps/unix -I../sysdeps/unix/sysv/linux/i386 -I../sysdeps/unix/sysv/linux -I../sysdeps/gnu -I../sysdeps/unix/common -I../sysdeps/unix/mman -I../sysdeps/unix/inet -I../sysdeps/unix/sysv/i386 -I../sysdeps/unix/sysv -I../sysdeps/unix/i386 -I../sysdeps/unix -I../sysdeps/posix -I../sysdeps/i386/i686/fpu -I../sysdeps/i386/i686 -I../sysdeps/i386/i486 -I../sysdeps/i386/fpu -I../sysdeps/i386 -I../sysdeps/wordsize-32 -I../sysdeps/ieee754/ldbl-96 -I../sysdeps/ieee754/dbl-64 -I../sysdeps/ieee754/flt-32 -I../sysdeps/ieee754 -I../sysdeps/generic/elf -I../sysdeps/generic -nostdinc -isystem /usr/lib/gcc/i686-pc-linux-gnu/3.4.4/include -isystem /usr/include -D_LIBC_REENTRANT -include ../include/libc-symbols.h -DPIC -DSHARED -D_IO_MTSAFE_IO -o /var/tmp/portage/glibc-2.3.5-r1/work/build-default-i686-pc-linux-gnu-linuxthreads/libio/iovsprintf.os -MD -MP -MF /var/tmp/portage/glibc-2.3.5-r1/work/build-default-i686-pc-linux-gnu-linuxthreads/libio/iovsprintf.os.dt -MT /var/tmp/portage/glibc-2.3.5-r1/work/build-default-i686-pc-linux-gnu-linuxthreads/libio/iovsprintf.os /var/tmp/portage/glibc-2.3.5-r1/temp/ccVYlMaD.s: Assembler messages: /var/tmp/portage/glibc-2.3.5-r1/temp/ccVYlMaD.s:4514: Error: unknown pseudo-op: `.uleb168' make[2]: *** [/var/tmp/portage/glibc-2.3.5-r1/work/build-default-i686-pc-linux-gnu-linuxthreads/libio/ioungetc.os] Error 1 make[2]: *** Waiting for unfinished jobs.... make[2]: Leaving directory `/tmp/portage/glibc-2.3.5-r1/work/glibc-2.3.5/libio' make[1]: *** [libio/subdir_lib] Error 2 make[1]: Leaving directory `/tmp/portage/glibc-2.3.5-r1/work/glibc-2.3.5' make: *** [all] Error 2 -------------------------------------------------- Portage 2.0.51.22-r2 (default-linux/x86/2005.0, gcc-3.4.4, glibc-2.3.5-r1, 2.6.12-gentoo-r6 i686) ================================================================= System uname: 2.6.12-gentoo-r6 i686 AMD Athlon(tm) processor Gentoo Base System version 1.6.13 dev-lang/python: 2.3.5, 2.4.1-r1 sys-apps/sandbox: 1.2.12 sys-devel/autoconf: 2.13, 2.59-r7 sys-devel/automake: 1.4_p6, 1.5, 1.6.3, 1.7.9-r1, 1.8.5-r3, 1.9.6 sys-devel/binutils: 2.16.1 sys-devel/libtool: 1.5.18-r1 virtual/os-headers: 2.6.11-r2 ACCEPT_KEYWORDS="x86 ~x86" AUTOCLEAN="yes" CBUILD="i686-pc-linux-gnu" CFLAGS="-ggdb3 -O0" CHOST="i686-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/kde/2/share/config /usr/kde/3/share/config /usr/lib/X11/xkb /usr/lib/fax /usr/share/config /var/qmail/alias /var/qmail/control /var/spool/fax/etc" CONFIG_PROTECT_MASK="/etc/gconf /etc/terminfo /etc/env.d" CXXFLAGS="-mtune=athlon-tbird -O2 -fomit-frame-pointer -pipe" DISTDIR="/usr/portage/distfiles" FEATURES="autoconfig distlocks sandbox sfperms strict" GENTOO_MIRRORS="http://ftp.belnet.be/mirror/rsync.gentoo.org/gentoo/ ftp://ftp.belnet.be/mirror/rsync.gentoo.org/gentoo/ http://pandemonium.tiscali.de/pub/gentoo/ http://194.117.143.71" LANG="german" LC_ALL="de_DE.utf8" MAKEOPTS="-j2" PKGDIR="/usr/portage/packages" PORTAGE_TMPDIR="/var/tmp" PORTDIR="/usr/portage" PORTDIR_OVERLAY="/usr/local/portage" SYNC="rsync://rsync.europe.gentoo.org/gentoo-portage" USE="x86 3dnow aalib acl acpi apache2 apm avi bash-completion berkdb bzlib cairo crypt cups curl dba dbx dedicated dhcp divx4linux dvd dvdread dynagraph encode ethereal fam fastcgi foomaticdb ftp gd gif glitz graphviz icq imagemagick imap imlib innodb ipv6 jpeg libcaca libclamav libwww lm_sensors mad maildir mailwrapper mime mmx mpeg mpm-worker mssql mysql mythtv ncurses network nls nocd nptl oav odbc oggvorbis opengl pam pcre pear-db perl php png pnp postgres ppds prelude python qmail quicktime readline real rrdtool samba session slang snmp soap sockets spamassassin spell spl ssl svg tcpd threads tidy tiff tokenizer truetype underscores unicode usb v4l v4l2 vhosts winbind wmf xml xml2 xsl xslt xv xvid zlib fritzcapi_cards_fcpci userland_GNU kernel_linux elibc_glibc" Unset: ASFLAGS, CTARGET, LDFLAGS, LINGUAS
Just a little status update: Both glibc and openssl didn't compile with these CFLAGS. openssl did with -O2 instead of -O0, but glibc still failes (with another error). I try to test qmail with the new openssl tomorrow, if I have some time.
The newly compiled openssl has brought no changes at all to the backtrace. glibc just doesn't compile with the -ggdb3 CFLAG. Is there any other thing that I could test?
I don't know much about x86, so please file separate bug for the glibc thing. Other than that, I'll try to reproduce the error.
I've been able to reproduce the crash: @4000000042fa882b1952b05c starting delivery 6: msg 3589176 to remote michael@taipan.lan @4000000042fa882b1952df3c status: local 0/10 remote 1/20 @4000000042fa882b2a99a1cc delivery 6: deferral: qmail-remote_crashed./ Now I just have to fix it
I've been able to reproduce the crash: @4000000042fa882b1952b05c starting delivery 6: msg 3589176 to remote michael@taipan.lan @4000000042fa882b1952df3c status: local 0/10 remote 1/20 @4000000042fa882b2a99a1cc delivery 6: deferral: qmail-remote_crashed./ Now I just have to fix it :-)
I'm glad to hear that. ;-) May I ask, what you did to reproduce the bug?
I configured one machine in my network to require authentication and another one that routes all mails trough the first machine. In smtproutes, I specified a user and password. The user and password have to be plain text, not encoded with base64 ("With -r15 I had my username and password in smtproutes base64 encoded [...]"). I'll continue to work on this issue this evening.
It's fixed in CVS. Can you test it, please?
It works. Thanks a lot!
I've updated the patch again, but it still works (tested myself). Closing this bug. Thanks for testing.
