cp does not work correctly within afs space if coreutils are compiled with acl use flag. Example: andrej@f9pc46 ~ $ cp ./a.out ./b.out cp: overwrite `./b.out'? y cp: listing attributes of Ld$
cp does not work correctly within afs space if coreutils are compiled with acl use flag. Example: andrej@f9pc46 ~ $ cp ./a.out ./b.out cp: overwrite `./b.out'? y cp: listing attributes of Ld$èLt$: Numerical result out of range The file is copied, but the return code of cp is 1, which breaks makefiles etc... I am not sure whether this is coreutils, attr-2.4.23 or openafs-1.4.0 problem. lsattr on nfs dir returns "Operation not supported While reading flags", on afs it returns "Invalid argument While reading flags"
some additional info: that happens on amd64 dual-core smp: Linux f9pc46 2.6.14-gentoo-r2 #2 SMP Wed Nov 16 09:22:36 CET 2005 x86_64 AMD Athlon(tm) 64 X2 Dual Core Processor 3800+ AuthenticAMD GNU/Linux but not on uniprocessor x86 with same package versions. on another amd64 (2.6.12-gentoo-r9, openafs-1.3.87) it works OK. If I copy cp, libattr, libacl and libc.so from that machine to the problematic machine it still does not work. Could that be a kernel problem? emerge --info of the problematic machine: Portage 2.0.53_rc7 (default-linux/amd64/2005.1, gcc-3.4.4, glibc-2.3.5-r3, 2.6.14-gentoo-r2 x86_64) ================================================================= System uname: 2.6.14-gentoo-r2 x86_64 AMD Athlon(tm) 64 X2 Dual Core Processor 3800+ Gentoo Base System version 1.12.0_pre10 distcc 2.18.3 x86_64-pc-linux-gnu (protocols 1 and 2) (default port 3632) [enabled] ccache version 2.4 [disabled] dev-lang/python: 2.3.5, 2.4.2 sys-apps/sandbox: 1.2.13 sys-devel/autoconf: 2.13, 2.59-r7 sys-devel/automake: 1.4_p6, 1.5, 1.6.3, 1.7.9-r1, 1.8.5-r3, 1.9.6-r1 sys-devel/binutils: 2.16.1 sys-devel/libtool: 1.5.20-r1 virtual/os-headers: 2.6.11-r3 ACCEPT_KEYWORDS="amd64 ~amd64" AUTOCLEAN="yes" CBUILD="x86_64-pc-linux-gnu" CFLAGS="-O2 -pipe" CHOST="x86_64-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/kde/2/share/config /usr/kde/3.4/env /usr/kde/3.4/share/config /usr/kde/3.4/shutdown /usr/kde/3/share/config /usr/lib/X11/xkb /usr/lib64/mozilla/defaults/pref /usr/share/config /var/bind /var/qmail/control" CONFIG_PROTECT_MASK="/etc/gconf /etc/terminfo /etc/texmf/web2c /etc/env.d" CXXFLAGS="-O2 -pipe" DISTDIR="/net/teran/d0/nfs/gentoo/distfiles" FEATURES="autoconfig buildpkg distcc distlocks sandbox sfperms strict" GENTOO_MIRRORS="http://ftp.gentoo.skynet.be/pub/gentoo http://gentoo.oregonstate.edu http://www.ibiblio.org/pub/Linux/distributions/gentoo" MAKEOPTS="-j6" PKGDIR="/net/teran/d0/nfs/gentoo/packages/amd64.general" PORTAGE_TMPDIR="/var/tmp" PORTDIR="/usr/portage" PORTDIR_OVERLAY="/net/teran/d0/nfs/gentoo/portage.local" SYNC="rsync://rsync.gentoo.org/gentoo-portage" USE="amd64 X Xaw3d a52 aac aalib accessibility accounting acl alsa apache2 arts athena atlas audiofile avi bash-completion berkdb bidi bitmap-fonts blas bonobo bzip2 cdparanoia cdr cli crypt cups curl dba dbm dbus dedicated dga directfb divx4linux dts dvd dvdr edl eds emacs emacs-w3 emboss encode esd ethereal exif expat fam fame fastcgi fbcon festival ffmpeg firefox flac font-server foomaticdb fortran gcj gd gdbm ggi gif gimpprint glut gmp gnome gnuplot gphoto2 gpm graphviz gstreamer gtk gtk2 gtkhtml guile hal howl idn ieee1394 imagemagick imap imlib innodb ipv6 irda ithreads jack java joystick jpeg junit kde kerberos krb4 lcms ldap libcaca libclamav libwww lirc live lm_sensors logrotate lzo lzw lzw-tiff mad matroska mbox mhash mikmod mime mng motif mozilla mozsvg mp3 mpeg mule musicbrainz mysql mysqli nagios-dns nagios-ntp nagios-ping nas ncurses network nls nptl nvidia objc odbc ogg openal openexr opengl oss pam pam_console pcmcia pcre pda pdflib pear perl perlsuid php png posix postgres postgresql povray ppds python qt quicktime quotas readline rtc samba sasl sdl session slang smime snmp sox speex spell ssl svg tcltk tcpd tetex tga theora threads tiff truetype truetype-fonts type1-fonts udev unicode usb v4l v4l2 vhosts vorbis webdav wmf xanim xfs xine xinerama xml xml2 xmms xpm xscreensaver xv xvid xvmc zlib userland_GNU kernel_linux elibc_glibc" Unset: ASFLAGS, CTARGET, LANG, LC_ALL, LDFLAGS, LINGUAS
Further info: strace of a command "cp a.out b.out" on 2.6.13 and 2.6.14 gives the following difference close(4) = 0 close(3) = 0 -llistxattr("a.out", (nil), 0) = -1 EOPNOTSUPP (Operation not supported) +llistxattr("a.out", (nil), 0) = 17 +llistxattr("a.out", 0x7fffff8c6cb0, 17) = -1 ERANGE (Numerical result out of range) +write(2, "./cp: ", 6./cp: ) = 6 +write(2, "listing attributes of L\211d$\350L\211", 29listing attributes of Ld$) = 29 +write(2, ": Numerical result out of range", 31: Numerical result out of range) = 31 +write(2, "\n", 1 +) = 1 close(1) = 0 llistxattr("a.out", (nil), 0) should return EOPNOTSUPP, but it returns 17. I have found the following difference in fs/xattr.c between 2.6.13 and 2.6.14 kernel which might be relevant: @@ -221,20 +235,24 @@ return -ENOMEM; } + error = security_inode_listxattr(d); + if (error) + goto out; error = -EOPNOTSUPP; if (d->d_inode->i_op && d->d_inode->i_op->listxattr) { - error = security_inode_listxattr(d); - if (error) - goto out; error = d->d_inode->i_op->listxattr(d, klist, size); - if (error > 0) { - if (size && copy_to_user(list, klist, error)) - error = -EFAULT; - } else if (error == -ERANGE && size >= XATTR_LIST_MAX) { - /* The file system tried to returned a list bigger - than XATTR_LIST_MAX bytes. Not possible. */ - error = -E2BIG; - } + } else { + error = security_inode_listsecurity(d->d_inode, klist, size); + if (size && error >= size) + error = -ERANGE; + } + if (error > 0) { + if (size && copy_to_user(list, klist, error)) + error = -EFAULT; + } else if (error == -ERANGE && size >= XATTR_LIST_MAX) { + /* The file system tried to returned a list bigger + than XATTR_LIST_MAX bytes. Not possible. */ + error = -E2BIG; } out: if (klist)
This isn't just a problem with afs. This is a problem with tmpfs as well. For example, try copying /dev/null somewhere. It'll fail with a similar error. I don't know enough about the whys of it, but something changed with the xattrs stuff in 2.6.14. Specifically the commit is at: http://www.kernel.org/git/?p=linux/kernel/git/gregkh/linux-2.6.14.y.git;a=commit;h=f549d6c18c0e8e6cf1bf0e7a47acc1daf7e2cec1 This is preventing me from emergeing KDE 3.4.3 as at some point it tries to copy /dev/null to a file and it fails with this error.
Looks like something is wrong with the kernel, rather than with OpenAFS. I'm afraid that's not my area of expertise...
I'm having trouble reproducing this. Can anyone help? I've enabled CONFIG_SECURITY and CONFIG_SECURITY_CAPABILITIES in my kernel (2.6.14-gentoo-r4 so it includes that patch). I have coreutils merged with USE=acl /dev is mounted as tmpfs If I do: # cd /dev # cp null null2 Then I get no errors. Is this what other people are finding? Is my configuration the same as yours?
By the way, I was trying that as root. Not sure if that matters, I also played around with a few other setups but am unable to get any problems with tmpfs.
same here. coreutils is emerged with acl use flag. CONFIG_SECURITY and CONFIG_SECURITY_CAPABILITIES are both enabled in the kernel 2.6.14-gentoo-r2 for me. substrate etc # cd /dev/ substrate dev # cp null null2 cp: listing attributes of `null': Numerical result out of range as root.
Created attachment 74288 [details, diff] Patch I still can't reproduce this. You guys are using SELINUX or some other security policy, right? Anyway, this looks like a simple bounds checking error. Please try this patch and let me know if it helps.
Well, this patch works for me. SELINUX is in kernel, but not configured. I guess something changed between 2.6.13 and 2.6.14, so maybe there should be an explicit selinux=0 for boot (or selinux disabled in kernel config). Anyway, tail of strace for copy in usual filesystem: --------------- close(4) = 0 close(3) = 0 llistxattr("/tmp/00000", (nil), 0) = 0 llistxattr("/tmp/00000", 0x7fffffbb3af0, 0) = 0 close(1) = 0 exit_group(0) = ? tail of strace for /dev/null (or afs): ----------------- close(4) = 0 close(3) = 0 llistxattr("/dev/null", (nil), 0) = 17 llistxattr("/dev/null", 0x7fffffeb2100, 17) = 17 lgetxattr("/dev/null", "security.selinux", 0x0, 0) = 10 lgetxattr("/dev/null", "security.selinux", "unlabeled", 10) = 10 lsetxattr("/tmp/000", "security.selinux", "unlabeled", 10, 0) = 0 close(1) = 0 exit_group(0) = ? So I guess, the bug is solved... Thanks
I'll try the patch tonight. I also have SELINUX compiled into the kernel, but never properly configured. I always wanted to mess with it, but just haven't gotten around to it.
Yeah, just having selinux compiled in is probably enough to reveal this bug. But the bug isn't in selinux, its in the xattr code as shown by the patch. I'll send this upstream now. Thanks for testing.
Accepted upstream, will include in next release
Fixed in gentoo-sources-2.6.14-r5 (genpatches 2.6.14-6) and linux 2.6.14.4
Hiding the spam.
