Maounting of NFS shares fails partly with: root@impala:/root(17)# mount caiman:/home_caiman /mnt/test2 mount.nfs: Cannot allocate memory root@impala:/root(18)# type mount.nfs mount.nfs is /sbin/mount.nfs root@impala:/root(19)# qfile -v /sbin/mount.nfs net-fs/nfs-utils-1.2.6 (/sbin/mount.nfs) root@impala:/root(20)# genlop -t nfs-utils | tail Sat Sep 3 02:06:22 2011 >>> net-fs/nfs-utils-1.2.4 merge time: 1 minute and 5 seconds. Sat Jan 14 13:20:26 2012 >>> net-fs/nfs-utils-1.2.5 merge time: 1 minute and 1 second. Tue May 22 10:55:46 2012 >>> net-fs/nfs-utils-1.2.6 merge time: 58 seconds. root@impala:/root(21)# mount | grep nfs rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw) nfsd on /proc/fs/nfsd type nfsd (rw,nodev,noexec,nosuid) caiman:/usr on /net/caiman/usr type nfs (rw,nosuid,nodev,intr,sloppy,vers=4,addr=192.168.1.86,clientaddr=192.168.1.81) root@impala:/root(23)# showmount -e caiman | grep /home_caiman /home_caiman wildcat.homenet,impala.homenet,lynx2.homenet,lynx.homenet,caiman.homenet,condor.homenet,vilm.homenet,grizzly.homenet,inspironM12.homenet,thinkpad.homenet,tarantel.homenet,laptop.homenet,STar.homenet,NewSTar.homenet,lionhomenet.homenet,laptop450.homenet,shark.homenet,cathomenet.homenet,mouse.homenet /home_caiman/ftp wildcat.homenet,impala.homenet,lynx2.homenet,lynx.homenet,caiman.homenet,condor.homenet,vilm.homenet,grizzly.homenet,inspironM12.homenet,thinkpad.homenet,tarantel.homenet,laptop.homenet,STar.homenet,NewSTar.homenet,lionhomenet.homenet,laptop450.homenet,shark.homenet,cathomenet.homenet,mouse.homenet Reproducible: Always root@impala:/root(24)# emerge --info Portage 2.1.11.47 (default/linux/amd64/10.0, gcc-4.6.3, glibc-2.16.0, 3.7.1-gentoo x86_64) ================================================================= System uname: Linux-3.7.1-gentoo-x86_64-AMD_Phenom-tm-_II_X4_965_Processor-with-gentoo-2.2 KiB Mem: 8165848 total, 184924 free KiB Swap: 50347696 total, 50100084 free Timestamp of tree: Mon, 21 Jan 2013 07:00:01 +0000 ld GNU ld (GNU Binutils) 2.23.1 app-shells/bash: 4.2_p42 dev-java/java-config: 2.1.12-r1 dev-lang/python: 2.7.3-r3, 3.2.3-r2 dev-util/cmake: 2.8.10.2-r1 dev-util/pkgconfig: 0.27.1 sys-apps/baselayout: 2.2 sys-apps/openrc: 0.11.8 sys-apps/sandbox: 2.6 sys-devel/autoconf: 2.13, 2.69 sys-devel/automake: 1.9.6-r3, 1.10.3, 1.11.6, 1.12.6, 1.13.1 sys-devel/binutils: 2.23.1 sys-devel/gcc: 4.6.3 sys-devel/gcc-config: 1.8 sys-devel/libtool: 2.4.2 sys-devel/make: 3.82-r4 sys-kernel/linux-headers: 3.7 (virtual/os-headers) sys-libs/glibc: 2.16.0 Repositories: gentoo x11 bicatali science sage-on-gentoo sunrise lisp java-overlay lordvan x-portage x-cpan g-octave ACCEPT_KEYWORDS="amd64 ~amd64" ACCEPT_LICENSE="* -@EULA PUEL Intel-SDP dlj-1.1 skype-eula skype-4.0.0.7-copyright googleearth AdobeFlash-10.3 cadsoft Oracle-BCLA-JavaSE MakeMKV-EULA" CBUILD="x86_64-pc-linux-gnu" CFLAGS="-march=amdfam10 -O2 -pipe" CHOST="x86_64-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/share/config /usr/share/gnupg/qualified.txt /usr/share/openvpn/easy-rsa /usr/share/polkit-1/actions /var/lib/hsqldb" CONFIG_PROTECT_MASK="${EPREFIX}/etc/gconf /etc/ca-certificates.conf /etc/dconf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/php/apache2-php5.4/ext-active/ /etc/php/cgi-php5.4/ext-active/ /etc/php/cli-php5.4/ext-active/ /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo /etc/texmf/language.dat.d /etc/texmf/language.def.d /etc/texmf/updmap.d /etc/texmf/web2c" CXXFLAGS="-march=amdfam10 -O2 -pipe" DISTDIR="/usr/portage/distfiles" FCFLAGS="-O2 -pipe" FEATURES="assume-digests binpkg-logs config-protect-if-modified distlocks ebuild-locks fixlafiles merge-sync news parallel-fetch protect-owned sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch xattr" FFLAGS="-O2 -pipe" GENTOO_MIRRORS="http://distfiles.gentoo.org" LANG="en_US.UTF-8" LDFLAGS="-Wl,-O1 -Wl,--as-needed" MAKEOPTS="-j5" PKGDIR="/usr/portage/packages" PORTAGE_CONFIGROOT="/" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages" PORTAGE_TMPDIR="/var/tmp" PORTDIR="/usr/portage" PORTDIR_OVERLAY="/var/lib/layman/x11 /var/lib/layman/bicatali /var/lib/layman/science /var/lib/layman/sage-on-gentoo /var/lib/layman/sunrise /var/lib/layman/lisp /var/lib/layman/java-overlay /var/lib/layman/lordvan /usr/local/portage /var/lib/cpan /var/lib/g-octave" SYNC="rsync://rsync.gentoo.org/gentoo-portage" USE="3dnow 3dnowext 64bit R X Xaw3d a52 aac accessibility acl acpi admin afs alsa amd64 ao apache2 apng applet archive arpack asf aspell assistant atlas audacious audiofile automap automount bash-completion berkdb blas blast bluetooth bluray boo boost bzip2 cairo cdda cddb cdf cdio cdparanoia cdr cg cgi chm cli consolekit corba cracklib crypt css cuda cups curl cxx daap db dbi dbm dbus declarative designer devhelp device-mapper dga dia dirac djvu doc dot dri ds2490 ds9097 ds9097u dts dv dvb dvd dvdr dvi dynamicplugin eds egl elf emacs emboss emf encode epiphany evo examples exif expat extensions extra extras faac faad fam ffmpeg fftw firefox fits flac fltk fontconfig foomaticdb fortran fortran95 fpm fpx fts3 fuse g3dvl gcj gd gdal gdbm gdu gedit geoip geolocation geos gfortran gif gimp git glade glib glpk gml gmp gnome gnome-keyring gnome-print gnuplot gnutls gold gphoto2 gpm grammar graphics graphtft graphviz grass gsl gsm gstreamer gtk gtk3 gudev guile harness hddtemp hdf hdf5 hdri html http httpd hvm hwdb iconv icq icu id3 id3tag ide imagemagick imap inotify introspection ipod ipv6 ithreads jabber jadetex java java6 jbig john jpeg jpeg2k kate kde kdepim kdrive kerberos keymap kpathsea kqemu kvm ladspa lame lapack latex lcms ldap lensfun libffi libkms libnotify libsamplerate live lm_sensors lua lzo mad mail maildir mapnik math matroska media-library mercurial mikmod mkl mmx mmxext mng mod modules mono motif mozilla mp3 mp4 mpeg mpi mplayer mtp mudflap multilib musepack musicbrainz mysql mysqli nautilus ncurses netcdf netpbm network networking networkmanager nfs nls nntp nptl nsplugin ntfs ntp numpy nvidia obex objc ocaml ocr octave odbc ofa ogdi ogg openexr opengl openmp overview pam pcre pda pdf perl plasma plotutils plugins png podcast policykit portaudio posix postgres postscript ppds preview-latex proj projectm projectx pstricks pulseaudio python python-bindings q16 q32 qemu qhull qt3support qt4 quicktime raw readline reiserfs reports rhythmbox rle romio rpc rrdcgi rrdtool sage samba sasl schroedinger science sdk sdl secure-delete semantic-desktop server session shout sip slang slp smart smbclient smp sms sndfile snmp soap sockets sound soup sox speex spell sql sqlite sse sse2 sse4a ssl stlport subtitles subversion sudo suexec svg svm swig szip t1lib tcl tcpd tex tex4ht texmacs tgif theora thesaurus threads thunderbird tidy tiff tk tools truetype udev udisks unicode usb userlocales utempter v4l vaapi vala vdpau video virtualbox visio vorbis vpx wav webdav webdav-serf webkit wmf wxwidgets x264 xa xattr xcb xemacs xetex xext xft xine xml xmlreader xmlrpc xpm xv xvid xvmc yaml youtube zlib zvbi" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mmap_emul mulaw multi null plug rate route share shm softvol" APACHE2_MODULES="actions alias auth_basic auth_digest authn_anon authn_dbd authn_core authn_dbm authn_default authn_file authz_core authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgid dav dav_fs dav_lock dbd deflate dir disk_cache env expires ext_filter file_cache filter headers ident imagemap include info lbmethod_byrequests log_config logio mem_cache mime mime_magic negotiation proxy proxy_ajp proxy_balancer proxy_connect proxy_http rewrite setenvif slotmem_shm so socache_shmcb speling status unique_id unixd userdir usertrack vhost_alias" CALLIGRA_FEATURES="kexi words flow plan sheets stage tables krita karbon braindump" CAMERAS="canon fuji ptp2" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" CURL_SSL="nss" DRACUT_MODULES="caps lvm mdraid syslog" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf superstar2 timing tsip tripmate tnt ubx" INPUT_DEVICES="keyboard mouse evdev" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" LINGUAS="de fr ru" NETBEANS_MODULES="apisupport cnd dlight enterprise ergonomics groovy gsf harness ide identity j2ee java mobility nb php profiler ruby websvccommon xml" PHP_TARGETS="php5-3" PYTHON_SINGLE_TARGET="python2_7" PYTHON_TARGETS="python2_7 python3_2" QEMU_SOFTMMU_TARGETS="arm i386 x86_64" QEMU_USER_TARGETS="arm i386 x86_64" RUBY_TARGETS="ruby18 ruby19" USERLAND="GNU" VIDEO_CARDS="nouveau" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account" USE_PYTHON="2.7 3.2" Unset: CPPFLAGS, CTARGET, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LC_ALL, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS
is there anything in `dmesg` ? what does `strace -f mount` show ?
Some NFS functionality is available, the /usr partition of "caiman" is mounted at "impala": root@impala:/root(30)# mount | grep nfs rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw) nfsd on /proc/fs/nfsd type nfsd (rw,nodev,noexec,nosuid) caiman:/usr on /net/caiman/usr type nfs (rw,nosuid,nodev,intr,sloppy,vers=4,addr=192.168.1.86,clientaddr=192.168.1.81) condor:/usr on /net/condor/usr type nfs (rw,nosuid,nodev,intr,sloppy,vers=4,addr=192.168.1.76,clientaddr=192.168.1.81) root@impala:/root(31)# ll /net/caiman/usr | head total 38992 drwxr-xr-x 3 root root 4096 Apr 2 2011 adm/ drwxr-xr-x 2 root root 204800 Jan 22 07:13 bin/ -rw-r--r-- 1 root root 2032 Aug 25 09:14 CDFLeapSeconds.txt drwxr-xr-x 3 root root 4096 Jan 17 2012 data/ drwxr-xr-x 3 root root 4096 Jan 24 2011 etc/ drwxr-xr-x 4 games root 4096 Aug 27 04:19 games/ drwxr-xr-x 6 root root 4096 Jan 24 2011 gnu-classpath-0.98/ drwxr-xr-x 717 root root 118784 Jan 22 06:39 include/ -rw-r--r-- 1 root root 10190 Aug 6 20:19 INSTALL I just found, that even the command 'mount caiman:/home_caiman /mnt/test2' works again (without any reboot of "impala" or "caiman", only 'emerge -uvDN world' was partly running). Thus, may only some bits in memory switchted randomly.
Now it is again impossible to access the partition /net/caiman/home_caiman, which was automounted four hours ago. I had to restart autofs (/etc/init.d/autofs restart) to access them again.
I hit almost the same error again: root@impala:/root(4)# mount caiman:/usr/local/portage/distfiles_caiman /mnt/test2 mount.nfs: Cannot allocate memory root@impala:/root(5)# strace -f -o strace_mount.txt mount caiman:/usr/local/portage/distfiles_caiman /mnt/test2 mount.nfs: Cannot allocate memory root@impala:/root(6)# dmesg | tail [ 255.393512] ata2.00: failed to get Identify Device Data, Emask 0x1 [ 255.393524] ata2.00: configured for UDMA/133 [ 255.393565] ata2: EH complete [ 255.475340] ata4.00: failed to get Identify Device Data, Emask 0x1 [ 255.508382] ata4.00: failed to get Identify Device Data, Emask 0x1 [ 255.508400] ata4.00: configured for UDMA/133 [ 255.508424] ata4: EH complete [ 256.312320] EXT4-fs (dm-6): re-mounted. Opts: user_xattr,commit=0 [ 256.318492] EXT4-fs (dm-10): re-mounted. Opts: user_xattr,commit=0 [ 483.646198] mount.nfs (28820) used greatest stack depth: 2168 bytes left In /var/log/messages I find: root@impala:/root(8)# tail /var/log/messages Jan 31 16:06:35 impala dbus[25017]: [system] Activated service 'org.freedesktop.NetworkManager' failed: Launch helper exited with unknown return code 1 Jan 31 16:07:37 impala kernel: [ 483.646198] mount.nfs (28820) used greatest stack depth: 2168 bytes left Jan 31 16:08:47 impala crontab[1032]: (root) LIST (root) Jan 31 16:09:47 impala su[28715]: Successful su for root by rose Jan 31 16:09:47 impala su[28715]: + /dev/pts/0 rose:root Jan 31 16:09:47 impala su[28715]: pam_unix(su:session): session opened for user root by rose(uid=1203) Jan 31 16:10:01 impala cron[1791]: (root) CMD (test -x /usr/sbin/run-crons && /usr/sbin/run-crons) Jan 31 16:10:01 impala run-crons[1808]: (root) CMD (/etc/cron.hourly/logsentry.cron) Jan 31 16:10:01 impala run-crons[1812]: (root) CMD (/etc/cron.hourly/vnstat) Jan 31 16:13:26 impala crontab[6894]: (root) LIST (root)
Created attachment 337456 [details] strace_mount.txt
I tried to run mount thought gdb: root@impala:/usr/portage(21)# grep ^CFLAGS /etc/make.conf CFLAGS="-march=amdfam10 -O2 -g" root@impala:/usr/portage(22)# grep ^FEAT /etc/make.conf FEATURES="nostrip noclean" root@impala:/usr/portage(23)# emerge -v1 nfs-utils root@impala:/usr/portage(22)# gdb /sbin/mount GNU gdb (Gentoo 7.5.1 p1) 7.5.1 Copyright (C) 2012 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-pc-linux-gnu". For bug reporting instructions, please see: <http://bugs.gentoo.org/>... Reading symbols from /sbin/mount.nfs...done. (gdb) set args caiman:/usr/local/portage/distfiles_caiman /mnt/test2 (gdb) run Starting program: /sbin/mount.nfs caiman:/usr/local/portage/distfiles_caiman /mnt/test2 warning: Could not load shared library symbols for linux-vdso.so.1. Do you need "set solib-search-path" or "set sysroot"? [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib64/libthread_db.so.1". mount.nfs: Cannot allocate memory [Inferior 1 (process 26770) exited with code 040] (gdb) set logging file backtrace.log (gdb) set logging on Copying output to backtrace.log. (gdb) bt No stack. (gdb) set logging off Done logging to backtrace.log. (gdb) quit root@impala:/usr/portage(35)# ll backtrace.log -rw-r--r-- 1 root root 20 Jan 31 16:57 backtrace.log root@impala:/usr/portage(36)# cat backtrace.log No stack. No stack. What I am doing wrong?
(In reply to comment #6) you have to set a breakpoint on exit. the code isn't getting killed or aborting, it's writing an error message and calling exit(). b *_exit b *exit r you might have better luck e-mailing linux-nfs@vger.kernel.org when it comes to debugging nfs behavior. i haven't gotten this low before w/nfs.
