Bug 166460 – net-dns/pdns-2.9.20-r1 (stable) segv's the child process with mysql backend

• Status: RESOLVED WORKSFORME
• Product: Gentoo Linux

[Description Nathan Sullivan 2007-02-12 04:37:21 UTC]

Installed and configured pdns and mysql fine, same configuration on an x86 box. Using a standard gmysql backend with a unix domain socket to mysql... segv's when connecting to the mysql backend for some reason...

Reproducible: Always

Steps to Reproduce:
I] dev-db/mysql 
     Available versions:  !3.23.58-r1 4.0.27-r1 4.1.22-r1 [M]~4.1.23_alpha20070101-r61 5.0.26-r2 ~5.0.32 [M]~5.0.34_alpha20070101-r61
     Installed:           5.0.26-r2(08:22:55 02/09/07)(berkdb big-tables -cluster debug -embedded extraengine -latin1 -max-idx-128 -minimal perl -selinux ssl -static)


[I] net-dns/pdns 
     Available versions:  2.9.20-r1 ~2.9.20-r2
     Installed:           2.9.20-r1(12:27:10 02/09/07)(debug -doc ldap mysql -postgres sqlite -static -tdb)
Actual Results:  
Feb 12 14:32:51 apollo-xtrint pdns[18958]: PowerDNS 2.9.20 (C) 2001-2006 PowerDNS.COM BV (Feb  9 2007, 12:26:22, gcc 3.4.4 (Gentoo 3.4.4-r1, ssp-3.4.4-1.0, pie-8.7.8)) starting up
Feb 12 14:32:51 apollo-xtrint pdns[18958]: PowerDNS comes with ABSOLUTELY NO WARRANTY. This is free software, and you are welcome to redistribute it according to the terms of the GPL version 2.
Feb 12 14:32:51 apollo-xtrint pdns[18958]: Set effective group id to 1004
Feb 12 14:32:51 apollo-xtrint pdns[18958]: Set effective user id to 101
Feb 12 14:32:51 apollo-xtrint pdns[18958]: Creating backend connection for TCP
Feb 12 14:32:51 apollo-xtrint pdns[18958]: Launched webserver on 202.3.101.212:16998
Feb 12 14:32:51 apollo-xtrint pdns[18958]: Master/slave communicator launching
Feb 12 14:32:51 apollo-xtrint pdns[18958]: gmysql Connection succesful
Feb 12 14:32:51 apollo-xtrint pdns[18958]: About to create 3 backend threads for UDP
Feb 12 14:32:51 apollo-xtrint pdns[18958]: Got a signal 11, attempting to print trace: 
Feb 12 14:32:51 apollo-xtrint pdns[18958]: gmysql Connection succesful
Feb 12 14:32:51 apollo-xtrint pdns[18958]: /usr/sbin/pdns_server-instance [0x46aeac]
Feb 12 14:32:51 apollo-xtrint pdns[18958]: /lib/libc.so.6 [0x2b19defcf580]
Feb 12 14:32:51 apollo-xtrint pdns[18958]: /usr/lib64/libmysqlclient.so.15(_db_return_+0xd4) [0x2b19df334de6]
Feb 12 14:32:51 apollo-xtrint pdns[18958]: /usr/lib64/libmysqlclient.so.15(mysql_read_default_options+0xdd) [0x2b19df335c99]
Feb 12 14:32:51 apollo-xtrint pdns[18958]: /usr/lib64/libmysqlclient.so.15(mysql_real_connect+0x541) [0x2b19df3383f0]
Feb 12 14:32:51 apollo-xtrint pdns[18958]: /usr/lib64/libgmysqlbackend.so(_ZN6SMySQLC1ERKSsS1_tS1_S1_S1_+0xc1) [0x2b19df1e4a07]
Feb 12 14:32:51 apollo-xtrint pdns[18958]: /usr/lib64/libgmysqlbackend.so(_ZN13gMySQLBackendC1ERKSsS1_+0x19b) [0x2b19df1dc8c3]
Feb 12 14:32:51 apollo-xtrint pdns[18958]: /usr/lib64/libgmysqlbackend.so(_ZN13gMySQLFactory4makeERKSs+0x35) [0x2b19df1e3499]
Feb 12 14:32:51 apollo-xtrint pdns[18958]: /usr/sbin/pdns_server-instance(_ZN17BackendMakerClass3allEv+0xe4) [0x45de5c]
Feb 12 14:32:51 apollo-xtrint pdns[18958]: /usr/sbin/pdns_server-instance(_ZN12UeberBackendC1ERKSs+0x14a) [0x478f0e]
Feb 12 14:32:51 apollo-xtrint pdns[18958]: /usr/sbin/pdns_server-instance(_ZN13PacketHandlerC1Ev+0x40) [0x4398ee]
Feb 12 14:32:51 apollo-xtrint pdns[18958]: /usr/sbin/pdns_server-instance(_ZN17CommunicatorClass8mainloopEv+0xe2) [0x4931f2]
Feb 12 14:32:51 apollo-xtrint pdns[18958]: /usr/sbin/pdns_server-instance(_ZN17CommunicatorClass12launchhelperEPv+0x9) [0x4a6cf5]
Feb 12 14:32:51 apollo-xtrint pdns[18958]: /lib/libpthread.so.0 [0x2b19dee8b39b]
Feb 12 14:32:51 apollo-xtrint pdns[18958]: /lib/libc.so.6(__clone+0x6e) [0x2b19df05c79e]
Feb 12 14:32:52 apollo-xtrint pdns[18855]: Our pdns instance (18958) exited after signal 6
Feb 12 14:32:52 apollo-xtrint pdns[18855]: Respawning

Expected Results:  
Successful backend connection, respond to requests.

Installation is acting as a dns master only, with slaves pulling from it.

[Comment 1 Nathan Sullivan 2007-02-12 04:38:37 UTC]

oops just noticed this machine had a 3.4.4 toolchain instead of 4.1 as id hoped it should have had. in the process of doing an emerge -ev world on it now with 4.1.x (stable), will advise if this helps...

[Comment 2 Jakub Moc (RETIRED) 2007-02-12 07:59:44 UTC]

Reopen if it doesn't work once you've recompiled all the stuff w/ the new gcc; mixing gcc versions does no good.

http://www.gentoo.org/doc/en/gcc-upgrading.xml

[Comment 3 Nathan Sullivan 2007-04-11 01:12:12 UTC]

after making sure everything is built with gcc 4.1.x again, still getting the below results:

Apr 11 11:10:00 apollo-xtrint pdns[12899]: All slave domains are fresh
Apr 11 11:10:00 apollo-xtrint pdns[12899]: gmysql Connection succesful
Apr 11 11:10:00 apollo-xtrint pdns[12899]: Got a signal 11, attempting to print trace: 
Apr 11 11:10:00 apollo-xtrint pdns[12899]: /usr/sbin/pdns_server-instance [0x46c6a6]
Apr 11 11:10:00 apollo-xtrint pdns[12899]: /lib/libc.so.6 [0x2b7154b9c690]
Apr 11 11:10:00 apollo-xtrint pdns[12899]: /usr/lib64/libmysqlclient.so.15(_db_return_+0xd2) [0x2b7154f00190]
Apr 11 11:10:00 apollo-xtrint pdns[12899]: /usr/lib64/libmysqlclient.so.15 [0x2b7154f03393]
Apr 11 11:10:00 apollo-xtrint pdns[12899]: /usr/lib64/libmysqlclient.so.15(mysql_real_query+0xbc) [0x2b7154f010fa]
Apr 11 11:10:00 apollo-xtrint pdns[12899]: /usr/lib64/libgmysqlbackend.so(_ZN6SMySQL7doQueryERKSs+0x135) [0x2b7154dadd6f]
Apr 11 11:10:00 apollo-xtrint pdns[12899]: /usr/lib64/libgmysqlbackend.so(_ZN6SMySQL7doQueryERKSsRSt6vectorIS2_ISsSaISsEESaIS4_EE+0x99) [0x2b7154dae12f]
Apr 11 11:10:00 apollo-xtrint pdns[12899]: /usr/sbin/pdns_server-instance(_ZN11GSQLBackend17getUpdatedMastersEPSt6vectorI10DomainInfoSaIS1_EE+0x7a) [0x4d19aa]
Apr 11 11:10:00 apollo-xtrint pdns[12899]: /usr/sbin/pdns_server-instance(_ZN12UeberBackend17getUpdatedMastersEPSt6vectorI10DomainInfoSaIS1_EE+0x24) [0x475efc]
Apr 11 11:10:00 apollo-xtrint pdns[12899]: /usr/sbin/pdns_server-instance(_ZN17CommunicatorClass17masterUpdateCheckEP13PacketHandler+0xc0) [0x48a786]
Apr 11 11:10:00 apollo-xtrint pdns[12899]: /usr/sbin/pdns_server-instance(_ZN17CommunicatorClass8mainloopEv+0x12d) [0x4902c7]
Apr 11 11:10:00 apollo-xtrint pdns[12899]: /usr/sbin/pdns_server-instance(_ZN17CommunicatorClass12launchhelperEPv+0x9) [0x4a6dc5]
Apr 11 11:10:00 apollo-xtrint pdns[12899]: /lib/libpthread.so.0 [0x2b7154a59027]
Apr 11 11:10:00 apollo-xtrint pdns[12899]: /lib/libc.so.6(__clone+0x6e) [0x2b7154c27f5e]
Apr 11 11:10:01 apollo-xtrint cron[12910]: (root) CMD (test -x /usr/sbin/run-crons && /usr/sbin/run-crons )

next step?

[Comment 4 Nathan Sullivan 2007-04-11 01:12:53 UTC]

oops forgot something...

Portage 2.1.2.2 (default-linux/amd64/2006.1, gcc-4.1.1, glibc-2.5-r0, 2.6.18-xen x86_64)
=================================================================
System uname: 2.6.18-xen x86_64 Intel(R) Xeon(R) CPU            5140  @ 2.33GHz
Gentoo Base System release 1.12.9
Timestamp of tree: Wed, 11 Apr 2007 00:30:08 +0000
ccache version 2.4 [enabled]
dev-lang/python:     2.4.3-r4
dev-python/pycrypto: 2.0.1-r5
dev-util/ccache:     2.4-r6
sys-apps/sandbox:    1.2.17
sys-devel/autoconf:  2.13, 2.61
sys-devel/automake:  1.4_p6, 1.5, 1.6.3, 1.7.9-r1, 1.8.5-r3, 1.9.6-r2, 1.10
sys-devel/binutils:  2.16.1-r3
sys-devel/gcc-config: 1.3.14
sys-devel/libtool:   1.5.22
virtual/os-headers:  2.6.17-r2
ACCEPT_KEYWORDS="amd64"
AUTOCLEAN="yes"
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-O2 -march=nocona -pipe"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc"
CONFIG_PROTECT_MASK="/etc/env.d /etc/gconf /etc/php/apache1-php5/ext-active/ /etc/php/apache2-php5/ext-active/ /etc/php/cgi-php5/ext-active/ /etc/php/cli-php5/ext-active/ /etc/revdep-rebuild /etc/terminfo"
CXXFLAGS="-O2 -march=nocona -pipe"
DISTDIR="/usr/portage/distfiles"
FEATURES="ccache distlocks metadata-transfer parallel-fetch sandbox sfperms strict"
GENTOO_MIRRORS="http://distfiles.gentoo.org http://distro.ibiblio.org/pub/linux/distributions/gentoo"
MAKEOPTS="-j3"
PKGDIR="/usr/portage/packages"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --delete-after --stats --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --filter=H_**/files/digest-*"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
PORTDIR_OVERLAY="/usr/local/portage-overlays/testing /usr/local/portage-overlays/layman/aross"
SYNC="rsync://rsync.gentoo.org/gentoo-portage"
USE="amd64 apache2 bash-completion berkdb big-tables bitmap-fonts bzip2 cli cracklib crypt cups curl curlwrappers dba debug dri dvd ethereal exif extraengine fam fortran ftp gd gdbm gpm iconv imagemagick imap innodb ipv6 isdnlog jpeg kerberos ldap libg++ lm_sensors logrotate mcal mhash midi mppe-mppc mysql mysqli ncurses nls nptl nptlonly pam pcntl pcre pear perl png posix ppds pppd python radius readline reflection ruby samba session simplexml snmp soap sockets spell spl sqlite ssl svg tcpd tidy tiff truetype truetype-fonts type1-fonts unicode usb userlocales utf8 vhosts wddx xml xml2 xmlrpc xorg xsl zip zlib" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mulaw multi null plug rate route share shm softvol" ELIBC="glibc" INPUT_DEVICES="keyboard mouse evdev" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" USERLAND="GNU" VIDEO_CARDS="apm ark ati chips cirrus cyrix dummy fbdev glint i128 i810 mga neomagic nv rendition s3 s3virge savage siliconmotion sis sisusb tdfx tga trident tseng v4l vesa vga via vmware voodoo"
Unset:  CTARGET, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LANG, LC_ALL, LDFLAGS, LINGUAS, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS

:)

[Comment 5 Sven Wegener 2007-04-14 23:01:56 UTC]

I'm using powerdns on a gcc-3.4.6 hardened amd64 system and on a gcc-4.1.2 non-hardened amd64 system and I'm not experiencing any problems.

Are you using any special powerdns configuration options? Looking at the backtraces powerdns logged the error has changed after you recompiled your system. Does the error changes between invocations of powerdns?

[Comment 6 Nathan Sullivan 2007-04-16 00:11:46 UTC]

generally what I notice is, powerdns starts fine (the daemon i think), but the child processes that load up are where the crash occurs, it seems to be failing on the mysql connectivity part (which is where all our records are stored), but the weird thing is after a while the children manage to start fine, i think they just respawn until they succeed and then all is good until next restart... can sometimes take 5mins total for all child processes to connect successfully without a crash...

[Comment 7 Nathan Sullivan 2007-04-17 23:05:06 UTC]

sven, are you using a mysql backend btw?

[Comment 8 Sven Wegener 2007-04-18 00:50:03 UTC]

yep, the one on hardened is using the gmysql backend. and for testing i also configured the non-hardened installation with a gmysql backend, without problem.

[Comment 9 Sven Wegener 2008-06-07 20:07:12 UTC]

Sorry, unable to reproduce.