Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 885793 - sys-apps/coreutils-9.1-r1: copy_file_range fails on CIFS, no fallback?
Summary: sys-apps/coreutils-9.1-r1: copy_file_range fails on CIFS, no fallback?
Status: RESOLVED FIXED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Current packages (show other bugs)
Hardware: All Linux
: Normal normal (vote)
Assignee: Gentoo's Team for Core System packages
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2022-12-13 16:14 UTC by Frank Limpert
Modified: 2023-06-22 07:19 UTC (History)
2 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
strace log of cp (cp-1.strace,6.61 KB, text/plain)
2022-12-13 16:14 UTC, Frank Limpert
Details
emerge --info sys-apps/coreutils (emerge-info-coreutils-9.1-r1.log,6.72 KB, text/x-log)
2022-12-15 10:59 UTC, Frank Limpert
Details
emerge --info sys-apps/coreutils (emerge-info-coreutils-9.1-r1.log,6.73 KB, text/x-log)
2022-12-15 11:11 UTC, Frank Limpert
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Frank Limpert 2022-12-13 16:14:14 UTC
Created attachment 842497 [details]
strace log of cp

On my systems the very essential cp and mv commands fail reproducibly copying large files (> 2GiB) from one cifs share to another.

strace-ing cp I found that no fall-back lecacy code is executed when copy_file_range returns an error. See the attached strace log for proof.

I would rather stay a little while longer on coreutils-8* on my productive servers until the new coreutils are more dependable. That's why I ask the package maintainers to mark coreutils-9.1-r1 unstable again.

I know how to mask package versions for myself. This issue is only about warning others.

Details to reproduce these errors:
sys-apps/coreutils-9.1-r1 [+acl, -caps]
sys-kernel/gentoo-sources-5.15.80 on x86_64
# CONFIG_FSCACHE is not set
CONFIG_CIFS=m
# CONFIG_CIFS_STATS2 is not set
CONFIG_CIFS_ALLOW_INSECURE_LEGACY=y
CONFIG_CIFS_UPCALL=y
CONFIG_CIFS_XATTR=y
CONFIG_CIFS_POSIX=y
CONFIG_CIFS_DEBUG=y
# CONFIG_CIFS_DEBUG2 is not set
# CONFIG_CIFS_DEBUG_DUMP_KEYS is not set
CONFIG_CIFS_DFS_UPCALL=y
CONFIG_CIFS_SWN_UPCALL=y

Running with enabled FSCACHE and cachefilesd did not help.
Comment 1 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2022-12-13 16:29:51 UTC
Could you report the bug upstream to coreutils at https://lists.gnu.org/mailman/listinfo/bug-coreutils then let us know once you have?

It's done by sending an email to mailto:bug-coreutils@gnu.org. Thanks.
Comment 2 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2022-12-13 16:32:18 UTC
can you also include emerge --info here?
Comment 3 Mike Gilbert gentoo-dev 2022-12-13 19:54:30 UTC
> copy_file_range(4, NULL, 5, NULL, 9223372035781033984, 0) = -1 ENOENT (No such file or directory)

In fact, cp does have fallback code in case copy_file_range fails; just not when it fails with errno = ENOENT. That seems rather unexpected, and might be called a kernel bug.

https://git.savannah.gnu.org/cgit/coreutils.git/tree/src/copy.c?h=v9.1#n268
Comment 4 Frank Limpert 2022-12-15 10:59:16 UTC
Created attachment 842865 [details]
emerge --info sys-apps/coreutils
Comment 5 Frank Limpert 2022-12-15 11:11:03 UTC
Created attachment 842867 [details]
emerge --info sys-apps/coreutils

extracted after emerge =sys-apps/coreutils-9.1-r1
Comment 6 Frank Limpert 2022-12-15 11:51:44 UTC
(In reply to Mike Gilbert from comment #3)
> > copy_file_range(4, NULL, 5, NULL, 9223372035781033984, 0) = -1 ENOENT (No such file or directory)
> 
> In fact, cp does have fallback code in case copy_file_range fails; just not
> when it fails with errno = ENOENT. That seems rather unexpected, and might
> be called a kernel bug.
> 
> https://git.savannah.gnu.org/cgit/coreutils.git/tree/src/copy.c?h=v9.1#n268

Indeed, after echo 7 > /proc/fs/cifs/cifsFYI I see

CIFS: Status code returned 0xc0000034 STATUS_OBJECT_NAME_NOT_FOUND

in the kernel log. It seems that copy_file_range in the cifs fs driver maps this to ENOENT.
Comment 7 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2022-12-17 20:38:14 UTC
(In reply to Frank Limpert from comment #6)
> (In reply to Mike Gilbert from comment #3)
> > > copy_file_range(4, NULL, 5, NULL, 9223372035781033984, 0) = -1 ENOENT (No such file or directory)
> > 
> > In fact, cp does have fallback code in case copy_file_range fails; just not
> > when it fails with errno = ENOENT. That seems rather unexpected, and might
> > be called a kernel bug.
> > 
> > https://git.savannah.gnu.org/cgit/coreutils.git/tree/src/copy.c?h=v9.1#n268
> 
> Indeed, after echo 7 > /proc/fs/cifs/cifsFYI I see
> 
> CIFS: Status code returned 0xc0000034 STATUS_OBJECT_NAME_NOT_FOUND
> 
> in the kernel log. It seems that copy_file_range in the cifs fs driver maps
> this to ENOENT.

Could you report a bug to the kernel folks then and let us know when you have? Thanks.
Comment 8 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2022-12-31 17:06:03 UTC
I've mentioned this on bug-coreutils at https://debbugs.gnu.org/60455 in case they consider it a bug on their side as well to not fallback, but this is still likely a kernel issue.
Comment 9 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2022-12-31 19:18:36 UTC
(In reply to Sam James from comment #8)
> I've mentioned this on bug-coreutils at https://debbugs.gnu.org/60455 in
> case they consider it a bug on their side as well to not fallback, but this
> is still likely a kernel issue.

The coreutils folks want to know if the cp was partial or not. Did any data get copied?
Comment 10 Frank Limpert 2023-01-06 20:47:47 UTC
(In reply to Sam James from comment #9)
> (In reply to Sam James from comment #8)
> > I've mentioned this on bug-coreutils at https://debbugs.gnu.org/60455 in
> > case they consider it a bug on their side as well to not fallback, but this
> > is still likely a kernel issue.
> 
> The coreutils folks want to know if the cp was partial or not. Did any data
> get copied?

An empty file was created:

# cp /mnt/Backup/EAV/data-eav-eav-aktiv-20221207.dump.xz /mnt/OldBackup/EAV/1
cp: error copying '/mnt/Backup/EAV/data-eav-eav-aktiv-20221207.dump.xz' to '/mnt/OldBackup/EAV/1/data-eav-eav-aktiv-20221207.dump.xz': No such file or directory
# stat /mnt/OldBackup/EAV/1/data-eav-eav-aktiv-20221207.dump.xz
  File: /mnt/OldBackup/EAV/1/data-eav-eav-aktiv-20221207.dump.xz
  Size: 0         	Blocks: 8          IO Block: 1048576 regular empty file
Device: 0,36	Inode: 81611419679  Links: 1
Access: (0664/-rw-rw-r--)  Uid: (    0/    root)   Gid: (   16/    cron)
Access: 2023-01-06 21:45:57.070743000 +0100
Modify: 2023-01-06 21:45:57.070743000 +0100
Change: 2023-01-06 21:45:57.070743000 +0100
 Birth: 2023-01-06 21:45:57.070743000 +0100
Comment 11 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2023-01-09 03:34:10 UTC
(In reply to Frank Limpert from comment #10)
> (In reply to Sam James from comment #9)
> > (In reply to Sam James from comment #8)
> > > I've mentioned this on bug-coreutils at https://debbugs.gnu.org/60455 in
> > > case they consider it a bug on their side as well to not fallback, but this
> > > is still likely a kernel issue.
> > 
> > The coreutils folks want to know if the cp was partial or not. Did any data
> > get copied?
> 
> An empty file was created:
> 
> # cp /mnt/Backup/EAV/data-eav-eav-aktiv-20221207.dump.xz /mnt/OldBackup/EAV/1
> cp: error copying '/mnt/Backup/EAV/data-eav-eav-aktiv-20221207.dump.xz' to
> '/mnt/OldBackup/EAV/1/data-eav-eav-aktiv-20221207.dump.xz': No such file or
> directory
> # stat /mnt/OldBackup/EAV/1/data-eav-eav-aktiv-20221207.dump.xz
>   File: /mnt/OldBackup/EAV/1/data-eav-eav-aktiv-20221207.dump.xz
>   Size: 0         	Blocks: 8          IO Block: 1048576 regular empty file
> Device: 0,36	Inode: 81611419679  Links: 1
> Access: (0664/-rw-rw-r--)  Uid: (    0/    root)   Gid: (   16/    cron)
> Access: 2023-01-06 21:45:57.070743000 +0100
> Modify: 2023-01-06 21:45:57.070743000 +0100
> Change: 2023-01-06 21:45:57.070743000 +0100
>  Birth: 2023-01-06 21:45:57.070743000 +0100

Thanks. coreutils has implemented a workaround: https://git.savannah.gnu.org/cgit/coreutils.git/commit/?id=7fc84d1c0f6b35231b0b4577b70aaa26bf548a7c.

Please report the issue to the kernel.org folks and let us know what happens.
Comment 12 Larry the Git Cow gentoo-dev 2023-03-14 19:48:55 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=cd0377a8949670625747ec22daf86f5bed7025d6

commit cd0377a8949670625747ec22daf86f5bed7025d6
Author:     Sam James <sam@gentoo.org>
AuthorDate: 2023-03-14 19:43:38 +0000
Commit:     Sam James <sam@gentoo.org>
CommitDate: 2023-03-14 19:48:33 +0000

    sys-apps/coreutils: add 9.1_p20230313 (unkeyworded, pre-release)
    
    Bug: https://bugs.gentoo.org/885793
    Signed-off-by: Sam James <sam@gentoo.org>

 sys-apps/coreutils/Manifest                       |   2 +
 sys-apps/coreutils/coreutils-9.1_p20230313.ebuild | 267 ++++++++++++++++++++++
 2 files changed, 269 insertions(+)
Comment 13 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2023-03-14 19:51:12 UTC
Frank, have you had a chance to report this upstream to kernel.org?
Comment 14 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2023-03-14 19:59:55 UTC
This has bitten KIO too.
Comment 15 Larry the Git Cow gentoo-dev 2023-03-23 16:47:40 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=6236777c9eb031d4637ea86950202248a1ab37eb

commit 6236777c9eb031d4637ea86950202248a1ab37eb
Author:     Andreas Sturmlechner <asturm@gentoo.org>
AuthorDate: 2023-03-23 16:45:15 +0000
Commit:     Andreas Sturmlechner <asturm@gentoo.org>
CommitDate: 2023-03-23 16:46:10 +0000

    kde-frameworks/kio: Also handle copy_file_range failing with ENOENT
    
    Bug: https://bugs.gentoo.org/885793
    Bug: https://bugs.gentoo.org/899706
    Signed-off-by: Andreas Sturmlechner <asturm@gentoo.org>

 ...0-handle-copy_file_range-failing-w-ENOENT.patch | 33 ++++++++
 kde-frameworks/kio/kio-5.104.0-r2.ebuild           | 94 ++++++++++++++++++++++
 2 files changed, 127 insertions(+)
Comment 16 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2023-06-22 07:19:45 UTC
Workaround is in >=9.2. Still needs to be reported upstream if you can though. Nothing more for us to do in Gentoo. Thanks!