when I try to emerge coretuils on a remote linux system, It always hangs at checking for sys/mntent.h... no checking for struct statfs.f_fstypename... no checking for library containing nanosleep... none required checking for working nanosleep... Reproducible: Always Steps to Reproduce: 1.emerge sys-apps/coreutils 2.wait for hang 3.profit! ;) Actual Results: checking for mntctl function and struct vmount... no checking for one-argument getmntent function... yes checking sys/mntent.h usability... no checking sys/mntent.h presence... no checking for sys/mntent.h... no checking for struct statfs.f_fstypename... no checking for library containing nanosleep... none required checking for working nanosleep... ^C (it hangs here until I interrupt it with ctrl-c) Exiting on signal 2 Expected Results: I expect it to emerge peacefully and without issue. Please note, this is a headless system.. I do *NOT* have any nvidia drivers installed gir config # emerge -s nvidia-drivers Searching... [ Results for search key : nvidia-drivers ] [ Applications found : 1 ] * x11-drivers/nvidia-drivers Latest version available: 180.29 Latest version installed: [ Not Installed ] Size of files: 51,046 kB Homepage: http://www.nvidia.com/ Description: NVIDIA X11 driver and GLX libraries License: NVIDIA gir config # gir config # emerge --info Portage 2.1.6.11 (default/linux/x86/2008.0, gcc-4.1.2, glibc-2.8_p20080602-r1, 2.6.25-gentoo-r7 i686) ================================================================= System uname: Linux-2.6.25-gentoo-r7-i686-Intel-R-_Xeon-R-_CPU_3050_@_2.13GHz-with-glibc2.0 Timestamp of tree: Fri, 01 May 2009 17:30:01 +0000 ccache version 2.4 [enabled] app-shells/bash: 3.2_p39 dev-lang/python: 2.4.4-r9, 2.5.4-r2 dev-python/pycrypto: 2.0.1-r6 dev-util/ccache: 2.4-r7 sys-apps/baselayout: 1.12.11.1 sys-apps/sandbox: 1.6-r2 sys-devel/autoconf: 2.13, 2.63 sys-devel/automake: 1.5, 1.7.9-r1, 1.8.5-r3, 1.9.6-r2, 1.10.2 sys-devel/binutils: 2.18-r3 sys-devel/gcc-config: 1.4.0-r4 sys-devel/libtool: 1.5.26 virtual/os-headers: 2.6.27-r2 ACCEPT_KEYWORDS="x86" CBUILD="i686-pc-linux-gnu" CFLAGS="-O2 -march=i686 -pipe" CHOST="i686-pc-linux-gnu" CONFIG_PROTECT="/etc /var/qmail/alias /var/qmail/control" CONFIG_PROTECT_MASK="/etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/php/apache2-php5/ext-active/ /etc/php/cgi-php5/ext-active/ /etc/php/cli-php5/ext-active/ /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo /etc/udev/rules.d" CXXFLAGS="-O2 -march=i686 -pipe" DISTDIR="/usr/portage/distfiles" FEATURES="ccache distlocks fixpackages parallel-fetch protect-owned sandbox sfperms strict unmerge-orphans userfetch" GENTOO_MIRRORS="http://mirrors.tds.net/gentoo ftp://ftp.gtlib.cc.gatech.edu/pub/gentoo" LDFLAGS="-Wl,-O1" MAKEOPTS="-j3" PKGDIR="/usr/portage/packages" PORTAGE_CONFIGROOT="/" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages" PORTAGE_TMPDIR="/var/tmp" PORTDIR="/usr/portage" SYNC="rsync://rsync.gentoo.org/gentoo-portage" USE="X acl apache2 berkdb bzip2 cli cracklib crypt ctype cups dri fortran ftp gd gdbm gpm gtk2 iconv imap isdnlog jpeg midi mudflap mysql ncurses nls nptl nptlonly openmp pam pcre perl php png pppd python qmail-spp qt readline reflection server session spl ssl suhosin sysfs tcpd truetype unicode x86 xml xorg zlib" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1 emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mmap_emul mulaw multi null plug rate route share shm softvol" APACHE2_MODULES="actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache dav dav_fs dav_lock dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" ELIBC="glibc" INPUT_DEVICES="keyboard mouse" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" USERLAND="GNU" VIDEO_CARDS="fbdev glint i810 intel mach64 mga neomagic nv r128 radeon savage sis tdfx trident vesa vga via vmware voodoo" Unset: CPPFLAGS, CTARGET, EMERGE_DEFAULT_OPTS, FFLAGS, INSTALL_MASK, LANG, LC_ALL, LINGUAS, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, PORTDIR_OVERLAY gir config #
download this attachment and rename it as test.c: http://bugs.gentoo.org/attachment.cgi?id=103507 then compile and run it like so: gcc test.c ./a.out see if that hangs forever as well
jason@gir ~/src $ ldd a.out linux-gate.so.1 => (0xffffe000) libc.so.6 => /lib/libc.so.6 (0xb7de2000) /lib/ld-linux.so.2 (0xb7f28000) jason@gir ~/src $ ./a.out and it just hangs there.. I have to hit ctrl-c to get it to stop.
so run it through strace. something is broken on your system if SIGALARM isnt being sent within a second or two.
heres the output of the strace.. jason@gir ~/src $ cat /tmp/debug execve("./a.out", ["./a.out"], [/* 35 vars */]) = 0 brk(0) = 0x804b000 access("/etc/ld.so.preload", R_OK) = -1 ENOENT (No such file or directory) open("/etc/ld.so.cache", O_RDONLY) = 3 fstat64(3, {st_mode=S_IFREG|0644, st_size=43367, ...}) = 0 mmap2(NULL, 43367, PROT_READ, MAP_PRIVATE, 3, 0) = 0xb7fb6000 close(3) = 0 open("/lib/libc.so.6", O_RDONLY) = 3 read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0pg\1\0004\0\0\0\374"..., 512) = 512 fstat64(3, {st_mode=S_IFREG|0755, st_size=1274604, ...}) = 0 mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb7fb5000 mmap2(NULL, 1283664, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb7e7b000 mmap2(0xb7faf000, 12288, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x133) = 0xb7faf000 mmap2(0xb7fb2000, 9808, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xb7fb2000 close(3) = 0 mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb7e7a000 set_thread_area({entry_number:-1 -> 6, base_addr:0xb7e7a6c0, limit:1048575, seg_32bit:1, contents:0, read_exec_only:0, limit_in_pages:1, seg_not_present:0, useable:1}) = 0 mprotect(0xb7faf000, 8192, PROT_READ) = 0 mprotect(0x8049000, 4096, PROT_READ) = 0 mprotect(0xb7fdc000, 4096, PROT_READ) = 0 munmap(0xb7fb6000, 43367) = 0 rt_sigaction(SIGALRM, {0x80484d4, [], 0}, NULL, 8) = 0 alarm(1) = 0 nanosleep({2147483647, 999999999}, 0x804a0ec) = ? ERESTART_RESTARTBLOCK (To be restarted) --- SIGINT (Interrupt) @ 0 (0) --- +++ killed by SIGINT +++ jason@gir ~/src $
make this change to the check_for_SIGALRM func: check_for_SIGALRM (int sig) { printf("sig(%i) vs SIGALRM(%i)\n", sig, SIGALRM); if (sig != SIGALRM) _exit (1); } then re-run the test. if you still dont see any output after a few seconds, try this simple program: #include <unistd.h> main() { alarm(1); while (1); } see if that exits after a second
truly weird.. Both the modified program and the simple program fail to exit after a second.. both must be killed with ctrl-c.. I tried them on 2 other gentoo boxes and it exits fine after a second. wtf?! jason@gir ~/src $ cat test2.c #include <unistd.h> main() { alarm(1); while (1); } jason@gir ~/src $ gcc -o test test2.c jason@gir ~/src $ strace ./test execve("./test", ["./test"], [/* 35 vars */]) = 0 brk(0) = 0x804b000 access("/etc/ld.so.preload", R_OK) = -1 ENOENT (No such file or directory) open("/etc/ld.so.cache", O_RDONLY) = 3 fstat64(3, {st_mode=S_IFREG|0644, st_size=43367, ...}) = 0 mmap2(NULL, 43367, PROT_READ, MAP_PRIVATE, 3, 0) = 0xb7f10000 close(3) = 0 open("/lib/libc.so.6", O_RDONLY) = 3 read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0pg\1\0004\0\0\0\374"..., 512) = 512 fstat64(3, {st_mode=S_IFREG|0755, st_size=1274604, ...}) = 0 mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb7f0f000 mmap2(NULL, 1283664, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb7dd5000 mmap2(0xb7f09000, 12288, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x133) = 0xb7f09000 mmap2(0xb7f0c000, 9808, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xb7f0c000 close(3) = 0 mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb7dd4000 set_thread_area({entry_number:-1 -> 6, base_addr:0xb7dd46c0, limit:1048575, seg_32bit:1, contents:0, read_exec_only:0, limit_in_pages:1, seg_not_present:0, useable:1}) = 0 mprotect(0xb7f09000, 8192, PROT_READ) = 0 mprotect(0x8049000, 4096, PROT_READ) = 0 mprotect(0xb7f36000, 4096, PROT_READ) = 0 munmap(0xb7f10000, 43367) = 0 alarm(1) = 0 -db
run `cat /proc/self/status` and post the output
jason@gir ~ $ cat /proc/self/status Name: cat State: R (running) Tgid: 2377 Pid: 2377 PPid: 15578 TracerPid: 0 Uid: 1000 1000 1000 1000 Gid: 1000 1000 1000 1000 FDSize: 256 Groups: 10 19 519 1000 1044 VmPeak: 1632 kB VmSize: 1632 kB VmLck: 0 kB VmHWM: 372 kB VmRSS: 372 kB VmData: 152 kB VmStk: 84 kB VmExe: 28 kB VmLib: 1340 kB VmPTE: 12 kB Threads: 1 SigQ: 0/32768 SigPnd: 0000000000000000 ShdPnd: 0000000000000000 SigBlk: 0000000000002000 SigIgn: 0000000000000000 SigCgt: 0000000000000000 CapInh: 0000000000000000 CapPrm: 0000000000000000 CapEff: 0000000000000000 voluntary_ctxt_switches: 2 nonvoluntary_ctxt_switches: 0 jason@gir ~ $
something in your environment is screwing this up. every sig mask there should be 0 across the board. but, we see: SigBlk: 0000000000002000 which means SIGALRM is being blocked for your process and every child. i suggest you run `pstree -p` and analyze the /proc/<pid>/status file of each parent process of your current shell (whose pid can be found by doing `echo $$`). figure out where SigBlk is first being screwed up.
soooo It looks like its coming from sshd somewhere.. I had them restart sshd (from the console of the server) but I still see the same thing.. it seems like the root sshd process is fine (not sigblk) gir ~ # ps auxw |grep sshd root 21343 0.0 0.0 4260 932 ? Ss 14:35 0:00 /usr/sbin/sshd root 21359 0.0 0.0 7100 2368 ? Ss 14:36 0:00 sshd: jason [priv] jason 21372 0.0 0.0 7232 1764 ? S 14:36 0:00 sshd: jason@pts/1 root 22498 0.0 0.0 1784 500 pts/1 R+ 15:03 0:00 grep --colour=auto sshd gir ~ # grep -i sigblk /proc/21343/status SigBlk: 0000000000000000 gir ~ # grep -i sigblk /proc/21359/status SigBlk: 0000000000002000 gir ~ # grep -i sigblk /proc/21372/status SigBlk: 0000000000002000 gir ~ # but all my spawned ssh processes have the signal block set.. any ideas?
are you using inetd/xinetd ? or just sshd directly ? if you have root, then `/etc/init.d/sshd restart` should work fine you could work around the problem by a simple program: #include <signal.h> #include <unistd.h> int main(int argc, char *argv[]) { sigset_t set; sigemptyset(&set); sigaddset(&set, SIGALRM); sigprocmask(SIG_UNBLOCK, &set, NULL); return execvp(argv[1], argv+1); } then do at runtime: exec ./a.out /bin/bash
right, ive tried restarting sshd many times and it still blocks my signals jason@gir ~ $ pstree -p | grep $$ |-sshd(18428)-+-sshd(18439)---sshd(18454)---bash(18462)-+-grep(18499) jason@gir ~ $ grep -i sigblk /proc/18454/stat stat statm status jason@gir ~ $ grep -i sigblk /proc/18454/status SigBlk: 0000000000002000 jason@gir ~ $ grep -i sigblk /proc/18439/status SigBlk: 0000000000002000 jason@gir ~ $ grep -i sigblk /proc/18428/status SigBlk: 0000000000000000 jason@gir ~ $ but I tried the workaround and was able to emerge coreutils finally! ;) so thanks for that!
aha! so i finally figured out where the hell this troublesome bug is coming from. I kept seeing it from time to time on different servers that I administer. I found a bug report https://bugzilla.redhat.com/show_bug.cgi?id=446103 that lead me to look at my tcp wrappers (/etc/hosts.allow) and It finally hit me! #---- BlockHosts Additions sshd: ALL: spawn (/usr/bin/blockhosts.py --verbose --echo "%c-%s" >> /var/log/blockhosts.log 2>&1 )& : allow I disabled this entry and then tried to ssh into the host, and then the issue was gone.. So its something funky with the way that blockhosts does its tcp-wrapper magic on the sshd.. I shall go bug them. -db ooh, for the record, blockhosts is at http://www.aczoom.com/blockhosts
should be all set now in the tree; thanks for the report! Commit message: Migrate to Debian patchset which includes a lot more patches http://sources.gentoo.org/sys-apps/tcp-wrappers/files/tcp-wrappers-7.6-headers.patch?rev=1.1 http://sources.gentoo.org/sys-apps/tcp-wrappers/files/tcp-wrappers-7.6-redhat-bug11881.patch?rev=1.1 http://sources.gentoo.org/sys-apps/tcp-wrappers/metadata.xml?r1=1.3&r2=1.4 http://sources.gentoo.org/sys-apps/tcp-wrappers/tcp-wrappers-7.6.22.ebuild?rev=1.1
