Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 329981 - dev-util/nvidia-cuda-toolkit fails to install (on btrfs?)
Summary: dev-util/nvidia-cuda-toolkit fails to install (on btrfs?)
Status: RESOLVED NEEDINFO
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: [OLD] Development (show other bugs)
Hardware: All Linux
: High major (vote)
Assignee: Michal Januszewski (RETIRED)
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2010-07-26 21:36 UTC by Claes
Modified: 2011-06-12 20:10 UTC (History)
2 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Claes 2010-07-26 21:36:43 UTC
The nvidia-cuda-toolkit fails to install, the failed install also fails to clean up after itself:

# emerge -a nvidia-cuda-toolkit

These are the packages that would be merged, in order:

Calculating dependencies... done!
[ebuild  N    ] dev-util/nvidia-cuda-toolkit-3.0  USE="-debugger -doc -opencl -profiler" 

Would you like to merge these packages? [Yes/No] 

>>> Verifying ebuild manifests

>>> Emerging (1 of 1) dev-util/nvidia-cuda-toolkit-3.0
 * cudatoolkit_3.0_linux_64_rhel5.3.run RMD160 SHA1 SHA256 size ;-) ...                                                                                                                                  [ ok ]
 * checking ebuild checksums ;-) ...                                                                                                                                                                     [ ok ]
 * checking auxfile checksums ;-) ...                                                                                                                                                                    [ ok ]
 * checking miscfile checksums ;-) ...                                                                                                                                                                   [ ok ]
 * CPV:  dev-util/nvidia-cuda-toolkit-3.0
 * REPO: gentoo
 * USE:  amd64 elibc_glibc kernel_linux multilib userland_GNU
>>> Unpacking source...
>>> Unpacking cudatoolkit_3.0_linux_64_rhel5.3.run to /var/tmp/portage/dev-util/nvidia-cuda-toolkit-3.0/work
12099+1 records in
206194+1 records out
105571556 bytes (106 MB) copied, 4.07855 s, 25.9 MB/s
>>> Source unpacked in /var/tmp/portage/dev-util/nvidia-cuda-toolkit-3.0/work
>>> Preparing source in /var/tmp/portage/dev-util/nvidia-cuda-toolkit-3.0/work ...
 * Applying nvidia-cuda-toolkit-3.0-enum_fix.patch ...                                                                                                                                                    [ ok ]
>>> Source prepared.
>>> Configuring source in /var/tmp/portage/dev-util/nvidia-cuda-toolkit-3.0/work ...
>>> Source configured.
>>> Compiling source in /var/tmp/portage/dev-util/nvidia-cuda-toolkit-3.0/work ...
>>> Source compiled.
>>> Test phase [not enabled]: dev-util/nvidia-cuda-toolkit-3.0

>>> Install nvidia-cuda-toolkit-3.0 into /var/tmp/portage/dev-util/nvidia-cuda-toolkit-3.0/image/ category dev-util
>>> Completed installing nvidia-cuda-toolkit-3.0 into /var/tmp/portage/dev-util/nvidia-cuda-toolkit-3.0/image/

ecompressdir: bzip2 -9 /opt/cuda/man                                                                                                                                                                            
making executable: opt/cuda/lib64/libcublas.so.3.0.14
making executable: opt/cuda/lib64/libcublasemu.so.3.0.14
making executable: opt/cuda/lib64/libcudart.so.3.0.14
making executable: opt/cuda/lib64/libcudartemu.so.3.0.14                                                                                                                                                        
making executable: opt/cuda/lib64/libcufft.so.3.0.14
making executable: opt/cuda/lib64/libcufftemu.so.3.0.14

>>> Installing (1 of 1) dev-util/nvidia-cuda-toolkit-3.0
!!! Failed to rename /opt/cuda/man/man3/.CU_DEVICE_ATTRIBUTE_REGISTERS_PER_BLOCK.3.bz2._portage_merge_.1746 to /opt/cuda/man/man3/CU_DEVICE_ATTRIBUTE_REGISTERS_PER_BLOCK.3.bz2
!!! [Errno 31] Too many links

>>> Failed to install dev-util/nvidia-cuda-toolkit-3.0, Log file:

>>>  '/var/tmp/portage/dev-util/nvidia-cuda-toolkit-3.0/temp/build.log'

 * GNU info directory index is up-to-date.

Reproducible: Always

Steps to Reproduce:
1. emerge nvidia-cuda-toolkit
2. fail
3. no profit! :(




I'm aware that btrfs is still considered experimental, but since it won't be much longer, (https://patchwork.kernel.org/patch/105744/) I think this bug is valid.

# mount | grep "on / "
/dev/sdc2 on / type btrfs (rw,noatime)

Possibly related to: http://thread.gmane.org/gmane.comp.file-systems.btrfs/3427

# emerge --info 
Portage 2.1.8.3 (default/linux/amd64/10.0/desktop/kde, gcc-4.4.3, glibc-2.11.2-r0, 2.6.34-gentoo-r1 x86_64)
=================================================================
System uname: Linux-2.6.34-gentoo-r1-x86_64-Intel-R-_Core-TM-2_Duo_CPU_E8400_@_3.00GHz-with-gentoo-1.12.13
Timestamp of tree: Mon, 26 Jul 2010 19:15:01 +0000
ccache version 2.4 [disabled]
app-shells/bash:     4.0_p37
dev-java/java-config: 2.1.11
dev-lang/python:     2.6.5-r2, 3.1.2-r3
dev-util/ccache:     2.4-r7
dev-util/cmake:      2.8.1-r2
sys-apps/baselayout: 1.12.13
sys-apps/sandbox:    1.6-r2
sys-devel/autoconf:  2.13, 2.65
sys-devel/automake:  1.9.6-r3, 1.10.3, 1.11.1
sys-devel/binutils:  2.20.1-r1
sys-devel/gcc:       4.3.4, 4.4.3-r2
sys-devel/gcc-config: 1.4.1
sys-devel/libtool:   2.2.6b
virtual/os-headers:  2.6.30-r1
ACCEPT_KEYWORDS="amd64"
ACCEPT_LICENSE="* -@EULA skype-eula AdobeFlash-10.1"
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-march=native -O2 -pipe"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/share/X11/xkb /usr/share/config /var/bind /var/lib/hsqldb"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/env.d/java/ /etc/fonts/fonts.conf /etc/gconf /etc/php/apache2-php5/ext-active/ /etc/php/cgi-php5/ext-active/ /etc/php/cli-php5/ext-active/ /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo"
CXXFLAGS="-march=native -O2 -pipe"
DISTDIR="/usr/portage/distfiles"
FEATURES="assume-digests collision-protect distlocks fixpackages multilib-strict news parallel-fetch protect-owned sandbox sfperms splitdebug strict unmerge-logs unmerge-orphans userfetch userpriv"
GENTOO_MIRRORS="http://mirror.mdfnet.se/mirror/gentoo ftp://mirror.mdfnet.se/gentoo http://ftp.ing.umu.se/linux/gentoo/ ftp://ftp.ing.umu.se/linux/gentoo/"
LDFLAGS="-Wl,--as-needed"
LINGUAS="se"
MAKEOPTS="-j3"
PKGDIR="/usr/portage/packages"
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
PORTDIR_OVERLAY="/usr/local/portage/layman/sunrise /usr/local/portage/layman/toolchain /usr/local/portage/layman"
SYNC="rsync://rsync.europe.gentoo.org/gentoo-portage"
USE="3dnow X a52 aac acl acpi alsa amd64 apm bash-completion branding bzip2 cairo cdr cli consolekit cracklib crypt cxx dbus dri dts dvd dvdr emboss encode exif fam firefox flac fortran gdbm gif gpm iconv ipv6 jpeg kde lcms ldap libnotify mad mikmod mmx mng modules mp3 mp4 mpeg mudflap multilib ncurses nptl nptlonly nsplugin ogg opengl openmp pam pango pcre pdf perl phonon png ppds pppd python qt3support qt4 readline reflection sdl session spell spl sse sse2 ssl startup-notification svg symlink sysfs tcpd tiff truetype unicode usb vorbis x264 xcb xml xorg xulrunner xv xvid zlib" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mmap_emul mulaw multi null plug rate route share shm softvol" APACHE2_MODULES="actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" ELIBC="glibc" INPUT_DEVICES="keyboard mouse evdev" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LINGUAS="se" NETBEANS_MODULES="harness ide nb php websvccommon" QEMU_SOFTMMU_TARGETS="i386 x86_64" QEMU_USER_TARGETS="i386 x86_64" RUBY_TARGETS="ruby18" USERLAND="GNU" VIDEO_CARDS="vesa nvidia" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account" 
Unset:  CPPFLAGS, CTARGET, EMERGE_DEFAULT_OPTS, FFLAGS, INSTALL_MASK, LANG, LC_ALL, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS
Comment 1 Jeroen Roovers (RETIRED) gentoo-dev 2010-07-26 21:47:05 UTC
Could be prepman that's failing.
Comment 2 Michal Januszewski (RETIRED) gentoo-dev 2010-07-26 23:43:04 UTC
"Too many links" is a filesystem error, and it seems to be originating from something Portage does at the merge stage.  The number of files in /opt/cuda/man/man3 is around 1k, so I wouldn't say its very excessive.

Adding dev-portage to CC to see whether they can provide any insight into this.
Comment 3 Zac Medico gentoo-dev 2010-07-27 00:10:51 UTC
From this is seems like a filesystem limitation or bug that could at best be worked around from the portage side (by skipping files that fail to merge):

http://opengroup.org/onlinepubs/007908775/xsh/rename.html
[EMLINK]
The file named by old is a directory, and the link count of the parent directory of new would exceed {LINK_MAX}.
Comment 4 Zac Medico gentoo-dev 2010-07-27 00:16:32 UTC
(In reply to comment #0)
> >>> Installing (1 of 1) dev-util/nvidia-cuda-toolkit-3.0
> !!! Failed to rename
> /opt/cuda/man/man3/.CU_DEVICE_ATTRIBUTE_REGISTERS_PER_BLOCK.3.bz2._portage_merge_.1746

Is this file a hardlink? You can get the number of links from 'ls -l' or the 'stat' command. It doesn't seem like hardlinks should cause rename to fail here, but maybe it gives a clue.
Comment 5 Claes 2010-08-04 20:55:41 UTC
# cd /opt/cuda/man/man3/
# ls -la | head -n12
total 101
drwxr-xr-x  1 root root 7746 Aug  4 22:50 .
drwxr-xr-x  1 root root   16 Aug  4 22:50 ..
-rw-r--r-- 88 root root   69 Aug  4 22:50 .CU_DEVICE_ATTRIBUTE_REGISTERS_PER_BLOCK.3.bz2._portage_merge_.7675
-rw-r--r--  1 root root 2774 Aug  4 22:50 CUCTX.3.bz2
-rw-r--r--  1 root root 2214 Aug  4 22:50 CUD3D10.3.bz2
-rw-r--r--  1 root root 3511 Aug  4 22:50 CUD3D10_DEPRECATED.3.bz2
-rw-r--r--  1 root root 2161 Aug  4 22:50 CUD3D11.3.bz2
-rw-r--r--  1 root root 2287 Aug  4 22:50 CUD3D9.3.bz2
-rw-r--r--  1 root root 3944 Aug  4 22:50 CUD3D9_DEPRECATED.3.bz2
-rw-r--r-- 88 root root   69 Aug  4 22:50 CUDA64_MEMCPY3D.3.bz2
-rw-r--r--  1 root root  855 Aug  4 22:50 CUDART.3.bz
# stat .CU_DEVICE_ATTRIBUTE_REGISTERS_PER_BLOCK.3.bz2._portage_merge_.7675 
  File: `.CU_DEVICE_ATTRIBUTE_REGISTERS_PER_BLOCK.3.bz2._portage_merge_.7675'
  Size: 69              Blocks: 0          IO Block: 4096   regular file
Device: ch/12d  Inode: 25710911    Links: 88
Access: (0644/-rw-r--r--)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 2010-08-04 22:50:22.000000000 +0200
Modify: 2010-08-04 22:50:22.000000000 +0200
Change: 2010-08-04 22:50:33.291596225 +0200
Comment 6 Claes 2010-08-04 21:01:26 UTC
# ls -la | grep "\-rw\-r\-\-r\-\- 88" | wc -l
88

I'm not sure what to make of this, but it seems like it could possibly be of interest.
Comment 7 Zac Medico gentoo-dev 2010-08-05 02:55:24 UTC
I wonder if your particular filesystem is broken. I use btrfs here, and /usr/libexec/git-core contains files that have 107 hardlinks. So, it seems that 88 hardlinks is not necessarily the maximum allowed under btrfs.
Comment 8 Claes 2010-08-05 09:09:03 UTC
88 hard links limit would be both low, and a strange number.

Is it the dev-vcs/git package that creates those links? I could try pulling that in and see what happens.
Comment 9 Zac Medico gentoo-dev 2010-08-05 09:10:09 UTC
Yes, dev-vcs/git.
Comment 10 Claes 2010-08-09 18:53:25 UTC
I already had git installed, but re-emerged, and that worked just fine.
It is ofc still possible that my fs is just plain broken.
Not sure what else I can say or do, aside from perhaps taking it to the btrfs mailing list.
Comment 11 Zac Medico gentoo-dev 2010-08-10 00:06:15 UTC
Maybe if you `rm -rf /opt/cuda` and re-install the package then it will just work.
Comment 12 Claes 2010-08-10 07:34:44 UTC
Unfortunately I've done that several times already.

Since portage dies on the link problem, the files are left there, and unless I remove the /opt/cuda directory I get conflicts.
Comment 13 Michal Januszewski (RETIRED) gentoo-dev 2011-06-11 20:39:21 UTC
Is this still an issue?
Comment 14 Claes 2011-06-12 18:49:32 UTC
I haven't done any cuda work for quite a long time, so actually I can't say.
Might as well close this, until someone eventually reports it again.
Comment 15 Michal Januszewski (RETIRED) gentoo-dev 2011-06-12 20:10:02 UTC
Ok, thanks, closing as NEEDINFO.