Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 635126 - sys-apps/portage: _reflink_linux_file_copy incorrectly interprets sendfile *offset parameter
Summary: sys-apps/portage: _reflink_linux_file_copy incorrectly interprets sendfile *o...
Status: RESOLVED FIXED
Alias: None
Product: Portage Development
Classification: Unclassified
Component: Core (show other bugs)
Hardware: All Linux
: Normal normal (vote)
Assignee: Portage team
URL:
Whiteboard:
Keywords:
Depends on:
Blocks: 635020 631448
  Show dependency tree
 
Reported: 2017-10-23 01:31 UTC by Zac Medico
Modified: 2018-02-01 23:56 UTC (History)
1 user (show)

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Zac Medico gentoo-dev 2017-10-23 01:31:40 UTC
The sendfile *offset parameter is supposed to reflect the read offset of the input file, and does not necessarily indicate that the same bytes were written to the output file. Therefore, _reflink_linux_file_copy should use the return value of sendfile to measure the actual number of bytes copied. This might fix bug 635002.
Comment 2 Kaylee 2017-10-23 05:41:06 UTC
The comment "For the copyfunc call, the fd_in file offset must be exactly equal to offset_out. The above do_lseek_data function guarantees correct state." (and another similar one later) seem to indicate that the "offset_out" variable is intended to be guaranteed to be synchronised with the read offset?
Comment 3 Zac Medico gentoo-dev 2017-10-23 06:44:36 UTC
(In reply to Kaylee from comment #2)
> The comment "For the copyfunc call, the fd_in file offset must be exactly
> equal to offset_out. The above do_lseek_data function guarantees correct
> state." (and another similar one later) seem to indicate that the
> "offset_out" variable is intended to be guaranteed to be synchronised with
> the read offset?

Yes, that's right. Since the sendfile documentation doesn't seem to guarantee that the number of bytes written will be equal to the amount read from the input file, I'm thinking that we may need to use lseek to ensure that the input file offset is in sync with the output file offset.
Comment 4 Larry the Git Cow gentoo-dev 2017-10-27 18:50:35 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/proj/portage.git/commit/?id=5b6cf172e378f6da88e9634aa4e89f2f34390659

commit 5b6cf172e378f6da88e9634aa4e89f2f34390659
Author:     Zac Medico <zmedico@gentoo.org>
AuthorDate: 2017-10-23 01:55:36 +0000
Commit:     Zac Medico <zmedico@gentoo.org>
CommitDate: 2017-10-27 18:38:02 +0000

    file_copy: use sendfile return value to measure bytes copied (bug 635126)
    
    The sendfile *offset parameter refers to the input file offest, so
    it cannot be used in the same way as the copy_file_range *off_out
    parameter. Therefore, add sf_wrapper function which implements the
    *off_out behavior for sendfile.
    
    Also update cfr_wrapper so that it does not rely on the fd_in file
    offset, and remove corresponding fd_in lseek calls which are no
    longer needed.
    
    The file offset of fd_in is now completely unused, except in the
    plain read/write loop, where lseek is called prior to entering
    the loop.
    
    Bug: https://bugs.gentoo.org/635126

 src/portage_util_file_copy_reflink_linux.c | 99 ++++++++++++++++++------------
 1 file changed, 59 insertions(+), 40 deletions(-)}