Two infra boxes started having ntpd play up lately; The details below are from a system with ECC memory, so I think it's a bug somewhere. ntp compiled with CFLAGS="-O2 -pipe -ggdb3" to get the trace below. GDB output from the core included: # /usr/sbin/ntpd -p /var/run/ntpd.pid -g Segmentation fault (core dumped) # gdb /usr/sbin/ntpd core ... Reading symbols from /usr/sbin/ntpd...Reading symbols from /usr/lib64/debug//usr/sbin/ntpd.debug...done. done. [New LWP 61886] [New LWP 61885] (gdb) thread apply all bt Thread 2 (Thread 0x733d009d4700 (LWP 61885)): #0 0x0000733cffe33a66 in pthread_join () from /lib64/libpthread.so.0 #1 0x00000c199c3dd6ff in my_pthread_warmup () at ntpd.c:329 #2 ntpdmain (argc=4, argv=0x7f9bec9af4a8) at ntpd.c:548 #3 0x00000c199c3cc0eb in main (argc=<optimized out>, argv=<optimized out>) at ntpd.c:382 Thread 1 (Thread 0x733d009e2700 (LWP 61886)): #0 0x0000733cff44e854 in ?? () from /usr/lib/gcc/x86_64-pc-linux-gnu/4.8.3/libgcc_s.so.1 #1 0x0000733cff44f2b4 in ?? () from /usr/lib/gcc/x86_64-pc-linux-gnu/4.8.3/libgcc_s.so.1 #2 0x0000733cff44f635 in ?? () from /usr/lib/gcc/x86_64-pc-linux-gnu/4.8.3/libgcc_s.so.1 #3 0x0000733cff4503e6 in ?? () from /usr/lib/gcc/x86_64-pc-linux-gnu/4.8.3/libgcc_s.so.1 #4 0x0000733cff450770 in _Unwind_ForcedUnwind () from /usr/lib/gcc/x86_64-pc-linux-gnu/4.8.3/libgcc_s.so.1 #5 0x0000733cffe39a93 in __pthread_unwind () from /lib64/libpthread.so.0 #6 0x0000733cffe30b89 in sigcancel_handler () from /lib64/libpthread.so.0 #7 <signal handler called> #8 0x0000733cffb36b2d in nanosleep () from /lib64/libc.so.6 #9 0x0000733cffb369c7 in sleep () from /lib64/libc.so.6 #10 0x00000c199c3dd4ba in my_pthread_warmup_worker (thread_args=<optimized out>) at ntpd.c:296 #11 0x0000733cffe3250d in start_thread () from /lib64/libpthread.so.0 #12 0x0000733cffb6af1d in clone () from /lib64/libc.so.6 (gdb) NTPD config: server 0.gentoo.pool.ntp.org server 1.gentoo.pool.ntp.org server 2.gentoo.pool.ntp.org server 3.gentoo.pool.ntp.org driftfile /var/lib/ntp/ntp.drift restrict default nomodify nopeer restrict -6 default nomodify nopeer restrict 127.0.0.1 restrict -6 ::1 tinker dispersion 1.00 panic 0 stepout 3 # emerge --info Portage 2.2.26 (python 3.3.5-final-0, hardened/linux/amd64, gcc-4.8.3, glibc-2.19-r1, 4.3.3-hardened-r7-infra30 x86_64) ================================================================= System uname: Linux-4.3.3-hardened-r7-infra30-x86_64-Intel-R-_Xeon-R-_CPU_E31270_@_3.40GHz-with-gentoo-2.2 KiB Mem: 8194864 total, 5253672 free KiB Swap: 4194300 total, 4194300 free Timestamp of repository gentoo: Mon, 15 Feb 2016 03:00:01 +0000 sh bash 4.3_p42-r1 ld GNU ld (Gentoo 2.24 p1.4) 2.24 app-shells/bash: 4.3_p42-r1::gentoo dev-lang/perl: 5.20.2::gentoo dev-lang/python: 2.7.10-r1::gentoo, 3.3.5-r1::gentoo, 3.4.1::gentoo dev-util/cmake: 3.2.2::gentoo dev-util/pkgconfig: 0.28-r1::gentoo sys-apps/baselayout: 2.2::gentoo sys-apps/openrc: 0.17::gentoo sys-apps/sandbox: 2.6-r1::gentoo sys-devel/autoconf: 2.69::gentoo sys-devel/automake: 1.13.4::gentoo, 1.14.1::gentoo, 1.15::gentoo sys-devel/binutils: 2.24-r3::gentoo sys-devel/gcc: 4.8.3::gentoo sys-devel/gcc-config: 1.7.3::gentoo sys-devel/libtool: 2.4.2-r1::gentoo sys-devel/make: 4.0-r1::gentoo sys-kernel/linux-headers: 4.3::gentoo (virtual/os-headers) sys-libs/glibc: 2.19-r1::gentoo Repositories: gentoo location: /usr/portage sync-type: rsync sync-uri: rsync://rsync.gentoo.org/gentoo-portage priority: -1000 infra-overlay location: /usr/local/infra-overlay masters: gentoo priority: 0 ACCEPT_KEYWORDS="amd64" ACCEPT_LICENSE="* -@EULA" CBUILD="x86_64-pc-linux-gnu" CFLAGS="-O2 -pipe" CHOST="x86_64-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/share/gnupg/qualified.txt" CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/dev.d /etc/env.d /etc/gconf /etc/gentoo-release /etc/init.d /etc/php/apache2-php5.5/ext-active/ /etc/php/cgi-php5.5/ext-active/ /etc/php/cli-php5.5/ext-active/ /etc/revdep-rebuild /etc/sandbox.d /etc/scsi_id.config /etc/terminfo /etc/udev" CXXFLAGS="-O2 -pipe" DISTDIR="/var/cache/portage/distfiles/" EMERGE_DEFAULT_OPTS="-q --color=n --newuse --selective=n --jobs 2 --load-average 8" FCFLAGS="-O2 -pipe" FEATURES="assume-digests binpkg-logs buildpkg collision-protect config-protect-if-modified distlocks ebuild-locks fixlafiles merge-sync multilib-strict parallel-fetch preserve-libs protect-owned sandbox sfperms split-elog split-log splitdebug strict unknown-features-warn unmerge-orphans userfetch userpriv usersandbox usersync xattr" FFLAGS="-O2 -pipe" GENTOO_MIRRORS="http://distfiles.gentoo.org" LANG="en_US.UTF-8" LDFLAGS="-Wl,-O1 -Wl,--as-needed" MAKEOPTS="-j 8 -l 8" PKGDIR="/var/cache/portage/packages/" PORTAGE_CONFIGROOT="/" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages" PORTAGE_TMPDIR="/var/tmp" USE="acl amd64 bash-completion berkdb bzip2 cli cracklib crypt cxx dri gdbm hardened iconv ipv6 jpeg justify libwww maildir mmx mmxext modules multilib ncurses nptl nptlonly openmp pam pax_kernel pcre perl pie png python readline seccomp session sse sse2 ssl ssp tcpd unicode urandom vim-syntax xattr xml xtpax zlib" ABI_X86="64" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="kexi words flow plan sheets stage tables krita karbon braindump author" CAMERAS="ptp2" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" CPU_FLAGS_X86="mmx mmxext sse sse2" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf superstar2 timing tsip tripmate tnt ublox ubx" INPUT_DEVICES="keyboard mouse evdev" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php5-5" PYTHON_SINGLE_TARGET="python2_7" PYTHON_TARGETS="python2_7 python3_3 python3_4" RUBY_TARGETS="ruby20" USERLAND="GNU" VIDEO_CARDS="fbdev glint intel mach64 mga nouveau nv r128 radeon savage sis tdfx trident vesa via vmware dummy v4l" XTABLES_ADDONS="account chaos checksum condition delude dhcpmac dnetmap echo fuzzy geoip gradm iface ipmark ipp2p ipset ipv4options length2 logmark lscan pknock psd quota2 rawnat steal sysrq tarpit tee" Unset: CC, CPPFLAGS, CTARGET, CXX, INSTALL_MASK, LC_ALL, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, USE_PYTHON
you're using old gcc/glibc there, and it looks like a canceled pthreads codepath, so you should really update those also on the topic, i assume you're only using ntp to keep the local server in sync rather than host the time for other servers. why not convert to tlsdate ? it's way simpler to set up, and it has seccomp support.
(In reply to SpanKY from comment #1) > you're using old gcc/glibc there, and it looks like a canceled pthreads > codepath, so you should really update those infra stays on stable things for a long time; I've told one of the boxes to upgrade to gcc-4.9.3/glibc-2.21 and we'll see what happens after that. Surprised that NOT updating gcc/glibc should have broken ntpd however. > also on the topic, i assume you're only using ntp to keep the local server > in sync rather than host the time for other servers. why not convert to > tlsdate ? it's way simpler to set up, and it has seccomp support. tlsdate doesn't have any monitoring/mitigation of local clock skew/rate-errors. Some of the host clocks are pretty bad without it (eg: turn off ntpd, and have a variable wrongness of 10 seconds/day).
(In reply to Robin Johnson from comment #2) well let us know if it fails w/current stable glibc/gcc. we don't really support older.
