When copying files with konqueror or krusader, the copied files are filled with zeroes. Filenames, sizes and directory structures (ie. when copying multiple files) are all correct, but the content of all files are completely zeroed out. This happens both when copying single files and multiple files. It also happens both when doing copy/paste and drag/drop methods. Copying files from the web does not corrupt files. Ie. dragging an image from a webpage to the desktop. Also copying files to/from sftp:/ using konqueror does not corrupt files. Copying files using the shell also does not corrupt files. After a fruitful talk with Philantrop in #gentoo-kde it became apparent that there's possibly something fishy in the kio_file slave at least on my box. Re-emerging kdelibs did not solve the problem. Reproducible: Always Steps to Reproduce: 1. Copy a file from homedir to the desktop using konqueror or krusader. 2. 3. Actual Results: File is copied. No errors show up in message-log or xsession errors. File contents is all zeroes. $ emerge --info Portage 2.1.2.7 (default-linux/amd64/2007.0, gcc-4.1.2, glibc-2.5-r2, 2.6.20-vs2.2.0-gentoo x86_64) ================================================================= System uname: 2.6.20-vs2.2.0-gentoo x86_64 AMD Opteron(tm) Processor 240 Gentoo Base System release 1.12.9 Timestamp of tree: Wed, 30 May 2007 18:30:01 +0000 ccache version 2.4 [enabled] dev-java/java-config: 1.3.7, 2.0.33-r1 dev-lang/python: 2.4.4-r4 dev-python/pycrypto: 2.0.1-r5 dev-util/ccache: 2.4-r7 sys-apps/sandbox: 1.2.17 sys-devel/autoconf: 2.13, 2.61 sys-devel/automake: 1.5, 1.6.3, 1.7.9-r1, 1.9.6-r2, 1.10 sys-devel/binutils: 2.16.1-r3 sys-devel/gcc-config: 1.3.16 sys-devel/libtool: 1.5.22 virtual/os-headers: 2.6.17-r2 ACCEPT_KEYWORDS="amd64 ~amd64" AUTOCLEAN="yes" CBUILD="x86_64-pc-linux-gnu" CFLAGS="-O2 -pipe" CHOST="x86_64-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/kde/3.5/env /usr/kde/3.5/share/config /usr/kde/3.5/shutdown /usr/share/X11/xkb /usr/share/config /var/bind" CONFIG_PROTECT_MASK="/etc/env.d /etc/env.d/java/ /etc/gconf /etc/revdep-rebuild /etc/splash /etc/terminfo" CXXFLAGS="-O2 -pipe" DISTDIR="/usr/portage/distfiles" FEATURES="buildpkg ccache distlocks metadata-transfer parallel-fetch sandbox sfperms strict" GENTOO_MIRRORS="http://distfiles.gentoo.org http://distro.ibiblio.org/pub/linux/distributions/gentoo" PKGDIR="/usr/portage/packages" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --delete-after --stats --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --filter=H_**/files/digest-*" PORTAGE_TMPDIR="/var/tmp" PORTDIR="/usr/portage" SYNC="rsync://rsync.gentoo.org/gentoo-portage" USE="X acl acpi alsa amd64 arts artswrappersuid bash-completion berkdb bitmap-fonts bzip2 cli cracklib crypt cups curl dbus dri fortran gdbm gif glitz gpm hal iconv ieee1394 imlib ipv6 isdnlog jpeg kde kdehiddenvisibility libg++ logitech-mouse midi mmx mng mp3 mppe-mppc mudflap mysql ncurses nls nptl nptlonly ogg opengl openmp pam pcre perl png pppd python qt3 rdesktop readline reflection samba session spl sse sse2 ssl svg tcpd tidy tiff truetype truetype-fonts type1-fonts unicode usb utempter vorbis xcomposite xine xorg xscreensaver zlib" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mulaw multi null plug rate route share shm softvol" ELIBC="glibc" INPUT_DEVICES="keyboard mouse evdev" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" USERLAND="GNU" VIDEO_CARDS="nvidia" Unset: CTARGET, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LANG, LC_ALL, LDFLAGS, LINGUAS, MAKEOPTS, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, PORTDIR_OVERLAY $ emerge -pv kdelibs konqueror These are the packages that would be merged, in order: Calculating dependencies... done! [ebuild R ] kde-base/kdelibs-3.5.7 USE="alsa arts cups kdehiddenvisibility tiff utempter -acl -avahi -branding -debug -doc -fam -jpeg2k -kdeenablefinal -kerberos -legacyssl -lua -openexr -spell -xinerama" 0 kB [ebuild R ] kde-base/konqueror-3.5.7 USE="arts debug kdehiddenvisibility -java -kdeenablefinal -xinerama" 0 kB Total: 2 packages (2 reinstalls), Size of downloads: 0 kB
This seems to be a problem with sendfile(). Setting use_sendfile = 0 (kdelibs/kioslave/file/file.cc:698) to prevent it from using sendfile on copy fixes the problem.
This is probably not kde's fault other than using sendfile() without detecting that it is not working properly (on my box anyway). I have source code to test sendfile() outside kde and it also corrupts the file. Is this really a kernel bug? glibc? something else?
Could you please try if this happens when using gentoo-sources?
Sure, I'll try. But isn't vserver-sources the same as gentoo-sources only with vserver-patches?
I guess the vserver patches screw up something. Without the patches sendfile fails (returns -1 and EINVAL) without trying to copy which kio_file would catch and automatically fall back to read/write copy scheme. Google turns up some notes on previous sendfile problems in gentoo vserver sources. Whoever is working with vserver-source should be notified of this asap since vserver-sources-2.2.0 have been marked as stable.
vserver, this is yours, I think.
works for me ...
This is caused by having sys-kernel/linux-headers-2.6.17-r2 installed. I emerged sys-kernel/linux-headers-2.6.20-r2 and recompiled my sendfile test code and the problem disappeared.
I screwed up my test code. The problem is still there on my box. :(
Created attachment 121718 [details] Testcode I've attached the code I'm using for testing. It attempts to copy the gentoo logo to a new file using sendfile and runs md5sum on both files after.
This has to be the deepest bug I've ever had to uncover. The reason I got a positive result yesterday was because I tested this on a different harddrive than my home dir where I usually test this. Different filesystem. This bug is filesystem related. My testcode (and sendfile) fails on XFS but is working on EXT2, EXT3 and JFS. I tested all four filesystems on LVM volumes. Apparently, vserver patched kernels doesn't catch sendfile failing on XFS like vanilla or gentoo patched kernels do.
Your test code fails here, too, on ext3 (EINVAL). I may be mistaken but I think your test code is wrong: int fddst = open( "./gentoo-copy.png", O_WRONLY | O_CREAT, initialMode ); [...] n = sendfile( fddst, fdsrc, &sf, MAX_IPC_SIZE ); But (from man 2 sendfile): Presently (Linux 2.6.9): in_fd, must correspond to a file which supports mmap(2)-like operations (i.e., it cannot be a socket); and out_fd must refer to a socket. [...] In Linux 2.4 and earlier, out_fd could refer to a regular file, and sendfile() changed the current offset of that file.
I was puzzled by that as well. The manual must be wrong because I'm basically mimicking kde's kio_file. KDE catches the EINVAL and falls back to read/write operations. But sendfile doesn't fail with EINVAL on LVM/XFS under 2.6.20-vs2.2.0-gentoo even though sendfile isn't doing it's job (as shown by the md5sum). Output on my gentoo box WITH vserver using ext3: $ uname -a && mount | grep mnt/hd && cd /mnt/hd/sendfile/ && gcc sendfile.c -o sendfile && ./sendfile && cd Linux proficio 2.6.20-vs2.2.0-gentoo #2 SMP PREEMPT Sun Jun 10 08:56:21 CEST 2007 x86_64 AMD Opteron(tm) Processor 240 AuthenticAMD GNU/Linux /dev/mapper/vg01-test on /mnt/hd type ext3 (rw) open ./gentoo.png unlink ./gentoo-copy.png create and open ./gentoo-copy.png attempting writing range: 0 - 32768 wrote: 32768 bytes attempting writing range: 32768 - 65536 wrote: 32768 bytes attempting writing range: 65536 - 98304 wrote: 32768 bytes attempting writing range: 98304 - 131072 wrote: 7783 bytes attempting writing range: 106087 - 138855 wrote: 0 bytes 974203a72122256495d1af7ee4b4211f ./gentoo.png 974203a72122256495d1af7ee4b4211f ./gentoo-copy.png Output on my gentoo box WITH vserver using xfs: $ uname -a && mount | grep mnt/hd && cd /mnt/hd/sendfile/ && gcc sendfile.c -o sendfile && ./sendfile && cd Linux proficio 2.6.20-vs2.2.0-gentoo #2 SMP PREEMPT Sun Jun 10 08:56:21 CEST 2007 x86_64 AMD Opteron(tm) Processor 240 AuthenticAMD GNU/Linux /dev/mapper/vg01-test on /mnt/hd type xfs (rw) open ./gentoo.png unlink ./gentoo-copy.png create and open ./gentoo-copy.png attempting writing range: 0 - 32768 wrote: 32768 bytes attempting writing range: 32768 - 65536 wrote: 32768 bytes attempting writing range: 65536 - 98304 wrote: 32768 bytes attempting writing range: 98304 - 131072 wrote: 7783 bytes attempting writing range: 106087 - 138855 wrote: 0 bytes 974203a72122256495d1af7ee4b4211f ./gentoo.png de370e596d933431e075eb6ede58c195 ./gentoo-copy.png Output on my gentoo box WITHOUT vserver using ext3: $ uname -a && mount | grep mnt/hd && cd /mnt/hd/sendfile/ && ls && gcc sendfile.c -o sendfile && ./sendfile && cd Linux proficio 2.6.20-gentoo-r8 #4 SMP PREEMPT Sun Jun 10 08:19:39 CEST 2007 x86_64 AMD Opteron(tm) Processor 240 AuthenticAMD GNU/Linux /dev/mapper/vg01-test on /mnt/hd type ext3 (rw) gentoo.png sendfile.c open ./gentoo.png unlink ./gentoo-copy.png create and open ./gentoo-copy.png attempting writing range: 0 - 32768 wrote: -1 bytes failed: Invalid argument 974203a72122256495d1af7ee4b4211f ./gentoo.png d41d8cd98f00b204e9800998ecf8427e ./gentoo-copy.png Output on my gentoo box WITHOUT vserver using xfs: $ uname -a && mount | grep mnt/hd && cd /mnt/hd/sendfile/ && ls && gcc sendfile.c -o sendfile && ./sendfile && cd Linux proficio 2.6.20-gentoo-r8 #4 SMP PREEMPT Sun Jun 10 08:19:39 CEST 2007 x86_64 AMD Opteron(tm) Processor 240 AuthenticAMD GNU/Linux /dev/mapper/vg01-test on /mnt/hd type xfs (rw) gentoo.png sendfile.c open ./gentoo.png unlink ./gentoo-copy.png create and open ./gentoo-copy.png attempting writing range: 0 - 32768 wrote: -1 bytes failed: Invalid argument 974203a72122256495d1af7ee4b4211f ./gentoo.png d41d8cd98f00b204e9800998ecf8427e ./gentoo-copy.png Output on my slackware box WITHOUT vserver using ext3: $ uname -a && mount | grep mnt/hd && cd /mnt/hd/sendfile/ && ls && gcc sendfile.c -o sendfile && ./sendfile && cd Linux t3 2.6.19.7 #3 SMP PREEMPT Fri Jun 8 12:31:14 CEST 2007 i686 i686 i386 GNU/Linux /dev/mapper/vg00-test on /mnt/hd type ext2 (rw) ./ ../ gentoo.png sendfile.c open ./gentoo.png unlink ./gentoo-copy.png create and open ./gentoo-copy.png attempting writing range: 0 - 32768 wrote: -1 bytes failed: Invalid argument 974203a72122256495d1af7ee4b4211f ./gentoo.png d41d8cd98f00b204e9800998ecf8427e ./gentoo-copy.png Output on my slackware box WITHOUT vserver using xfs: $ uname -a && mount | grep mnt/hd && cd /mnt/hd/sendfile/ && ls && gcc sendfile.c -o sendfile && ./sendfile && cd Linux t3 2.6.19.7 #3 SMP PREEMPT Fri Jun 8 12:31:14 CEST 2007 i686 i686 i386 GNU/Linux /dev/mapper/vg00-test on /mnt/hd type xfs (rw) ./ ../ gentoo.png sendfile.c open ./gentoo.png unlink ./gentoo-copy.png create and open ./gentoo-copy.png attempting writing range: 0 - 32768 wrote: -1 bytes failed: Invalid argument 974203a72122256495d1af7ee4b4211f ./gentoo.png d41d8cd98f00b204e9800998ecf8427e ./gentoo-copy.png
This is clearly not a KDE issue so we're out of here.
Created attachment 122340 [details, diff] Patch to add nosendfile useflag Other than vserver's daniel_hozac acknowledging the problem, I don't really know if something is actually being done to fix it. In the meantime I came up with a patch to disable the use of sendfile in kdelibs. I've been using it for a few days now and doesn't seem to break anything on my AMD64 box. It does make my KDE experience a whole lot better though. :)
can you please try with vserver-sources-2.2.0.4 and reopen if it still doesn't work
Same problem exists in 2.6.22-vs2.2.0.4-gentoo + kde-3.5.8.
Fixed in 2.6.22-vs2.2.0.5-gentoo.
