bitcoind daemon periodically segfaults on my server. I'm not sure, where is the bug, in hardened-sources, bitcoind or db. From dmesg: [ 7284.535411] bitcoind[2567]: segfault at 397035c153e ip 00000394030d7721 sp 00000393edff9b40 error 4 in libdb_cxx-4.8.so[39402f8d000+1ac000] [ 7284.535439] grsec: Segmentation fault occurred at 00000397035c153e in /usr/bin/bitcoind[bitcoind:2567] uid/euid:107/107 gid/egid:993/993, parent /sbin/init[init:1] uid/euid:0/0 gid/egid:0/0 [52336.964193] bitcoind[19195]: segfault at 38be5b786c8 ip 000003594057b0ce sp 000003592b7fc7a0 error 4 in libdb_cxx-4.8.so[35940536000+1ad000] [52336.964209] grsec: From xxx.xxx.xxx.xxx: Segmentation fault occurred at 0000038be5b786c8 in /usr/bin/bitcoind[bitcoind:19195] uid/euid:107/107 gid/egid:993/993, parent /sbin/init[init:1] uid/euid:0/0 gid/egid:0/0 [401265.515705] traps: bitcoind[28929] general protection ip:331de22c1c3 sp:331c8ff8120 error:0 in libdb_cxx-4.8.so[331de0dd000+1ad000] [401265.543992] grsec: From xxx.xxx.xxx.xxx: Segmentation fault occurred at (nil) in /usr/bin/bitcoind[bitcoind:28929] uid/euid:107/107 gid/egid:993/993, parent /sbin/init[init:1] uid/euid:0/0 gid/egid:0/0 [611759.535576] traps: bitcoind[23301] general protection ip:2eeccdc9f91 sp:2eebb7fcb10 error:0 in libdb_cxx-4.8.so[2eeccc7d000+1ad000] [611759.570750] grsec: From xxx.xxx.xxx.xxx: Segmentation fault occurred at (nil) in /usr/bin/bitcoind[bitcoind:23301] uid/euid:107/107 gid/egid:993/993, parent /sbin/init[init:1] uid/euid:0/0 gid/egid:0/0 Reproducible: Always $ emerge --info bitcoind Portage 2.1.11.55 (hardened/linux/amd64, gcc-4.7.2, glibc-2.15-r3, 3.8.6-hardened x86_64) ================================================================= System Settings ================================================================= System uname: Linux-3.8.6-hardened-x86_64-QEMU_Virtual_CPU_version_1.0-with-gentoo-2.2 KiB Mem: 1015972 total, 124084 free KiB Swap: 1999868 total, 1944420 free Timestamp of tree: Mon, 15 Apr 2013 00:45:01 +0000 ld GNU ld (GNU Binutils) 2.22 app-shells/bash: 4.2_p37 dev-lang/python: 2.7.3-r3, 3.2.3-r2 dev-util/cmake: 2.8.9 dev-util/pkgconfig: 0.28 sys-apps/baselayout: 2.2 sys-apps/openrc: 0.11.8 sys-apps/sandbox: 2.5 sys-devel/autoconf: 2.13, 2.69 sys-devel/automake: 1.11.6, 1.12.6 sys-devel/binutils: 2.22-r1 sys-devel/gcc: 4.7.2-r1 sys-devel/gcc-config: 1.7.3 sys-devel/libtool: 2.4-r1 sys-devel/make: 3.82-r4 sys-kernel/linux-headers: 3.7 (virtual/os-headers) sys-libs/glibc: 2.15-r3 Repositories: gentoo x-puleglot-overlay x-local ACCEPT_KEYWORDS="amd64" ACCEPT_LICENSE="* -@EULA" CBUILD="x86_64-pc-linux-gnu" CFLAGS="-O2 -pipe" CHOST="x86_64-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/share/gnupg/qualified.txt /usr/share/openvpn/easy-rsa" CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo" CXXFLAGS="-O2 -pipe" DISTDIR="/usr/portage/distfiles" EMERGE_DEFAULT_OPTS="--with-bdeps y" FCFLAGS="-O2 -pipe" FEATURES="assume-digests binpkg-logs config-protect-if-modified distlocks ebuild-locks fixlafiles merge-sync news parallel-fetch protect-owned sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync webrsync-gpg xattr" FFLAGS="-O2 -pipe" GENTOO_MIRRORS="http://de-mirror.org/gentoo/ ftp://de-mirror.org/gentoo/" LANG="ru_RU.utf8" LDFLAGS="-Wl,-O1 -Wl,--as-needed" MAKEOPTS="-j1" PKGDIR="/usr/portage/packages" PORTAGE_COMPRESS="xz" PORTAGE_CONFIGROOT="/" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages" PORTAGE_TMPDIR="/var/tmp" PORTDIR="/usr/portage" PORTDIR_OVERLAY="/var/lib/layman/puleglot-overlay /usr/portage/local" SYNC="" USE="acl aio amd64 bash-completion berkdb bzip2 caps cli cracklib crypt cxx device-mapper dri gdbm gpm hardened iconv icu idn ipv6 justify logrotate lzma maildir mmx modules mudflap multilib mysql ncurses nls nptl openmp pam pax_kernel pcre perl postgres python readline sasl session sse sse2 ssl systemd tcpd tls unicode urandom vim-syntax xattr zlib" ABI_X86="64" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mmap_emul mulaw multi null plug rate route share shm softvol" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="kexi words flow plan sheets stage tables krita karbon braindump" CAMERAS="ptp2" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" DRACUT_MODULES="lvm systemd" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf superstar2 timing tsip tripmate tnt ubx" GRUB_PLATFORMS="pc" INPUT_DEVICES="keyboard mouse evdev" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" LINGUAS="en ru" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php5-3" PYTHON_SINGLE_TARGET="python2_7" PYTHON_TARGETS="python2_7 python3_2" RUBY_TARGETS="ruby18 ruby19" USERLAND="GNU" VIDEO_CARDS="fbdev glint intel mach64 mga nouveau nv r128 radeon savage sis tdfx trident vesa via vmware dummy v4l" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account" Unset: CPPFLAGS, CTARGET, INSTALL_MASK, LC_ALL, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, USE_PYTHON ================================================================= Package Settings ================================================================= net-p2p/bitcoind-0.6.3 was built with the following: USE="eligius logrotate (multilib) -examples -upnp" ABI_X86="64" CFLAGS="-O0 -pipe -fno-omit-frame-pointer -g3 -ggdb3"
Created attachment 345644 [details] backtrace Output from command gdb -batch -ex "thread apply all bt full" /usr/bin/bitcoind /var/lib/bitcoin/core
$ emerge -qpv sys-libs/db [ebuild R ] sys-libs/db-4.8.30 USE="cxx -doc -examples -java -tcl {-test}"
I can't remember for sure, but seems two noticable things happened before this problem first appeared: - gcc upgrade 4.6.3 -> 4.7.2 and the world was fully reemerged - hardened sources upgrade 3.7* -> 3.8* (I actually tried all versions >=3.8.3)
We've progressed quite a bit. Are you still getting this issue?
(In reply to Alexander Tsoy from comment #3) > - gcc upgrade 4.6.3 -> 4.7.2 and the world was fully reemerged Okay I think I know what's going on here and why I can't reproduce this. Its related to bug #513386. The c++ abi changed on upgrading gcc and this broke against bitcoind-0.6.3 which was still comiled with 4.6. Its a problem we're starting to recognize in gentoo. For now, just recompile bitcoind. In the future, we're working on having portage recognize this mismatch and trigger a rebuild.
(In reply to Anthony Basile from comment #4) > We've progressed quite a bit. Are you still getting this issue? I haven't used bitcoind for a long time, so I can't say for sure if this problem still exist or not. :) (In reply to Anthony Basile from comment #5) > (In reply to Alexander Tsoy from comment #3) > > - gcc upgrade 4.6.3 -> 4.7.2 and the world was fully reemerged > > Okay I think I know what's going on here and why I can't reproduce this. > Its related to bug #513386. The c++ abi changed on upgrading gcc and this > broke against bitcoind-0.6.3 which was still comiled with 4.6. Its a > problem we're starting to recognize in gentoo. As I mentioned in comment #3 I reemerged the whole @world after gcc upgrade.
