Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 916601 - sys-apps/portage: testAsyncFunctionStdin incompatible with multiprocessing spawn
Summary: sys-apps/portage: testAsyncFunctionStdin incompatible with multiprocessing spawn
Status: RESOLVED FIXED
Alias: None
Product: Portage Development
Classification: Unclassified
Component: Core (show other bugs)
Hardware: All Linux
: Normal normal
Assignee: Portage team
URL:
Whiteboard:
Keywords: InVCS
Depends on: 916256
Blocks: 914876
  Show dependency tree
 
Reported: 2023-11-01 04:57 UTC by Zac Medico
Modified: 2024-02-03 22:54 UTC (History)
1 user (show)

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Zac Medico gentoo-dev 2023-11-01 04:57:13 UTC
The fd_pipes support from bug 915896 doesn't work for testAsyncFunctionStdin because _send_fd_pipes does run until after the test case has already restored the real stdin file descriptor. The test case can simply wait until after the subprocess exits before it restores the real stdin file descriptor.
Comment 1 Larry the Git Cow gentoo-dev 2023-11-02 14:57:18 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/proj/portage.git/commit/?id=3b1234ba69a31709cd5aec1ae070901e3a28bb7c

commit 3b1234ba69a31709cd5aec1ae070901e3a28bb7c
Author:     Zac Medico <zmedico@gentoo.org>
AuthorDate: 2023-11-01 05:07:32 +0000
Commit:     Zac Medico <zmedico@gentoo.org>
CommitDate: 2023-11-02 14:56:37 +0000

    testAsyncFunctionStdin: multiprocessing spawn compat
    
    For compatibility with the multiprocessing spawn start
    method, we delay restoration of the stdin file descriptor,
    since this file descriptor is sent to the subprocess
    asynchronously.
    
    Signed-off-by: Zac Medico <zmedico@gentoo.org>
    Bug: https://bugs.gentoo.org/916601

 lib/portage/tests/process/test_AsyncFunction.py | 30 +++++++++++++++++++------
 1 file changed, 23 insertions(+), 7 deletions(-)
Comment 2 Larry the Git Cow gentoo-dev 2023-11-06 21:42:13 UTC
The bug has been closed via the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=0baf5e0bd0879f55a87111ced2140262dbec689b

commit 0baf5e0bd0879f55a87111ced2140262dbec689b
Author:     Sam James <sam@gentoo.org>
AuthorDate: 2023-11-06 21:41:33 +0000
Commit:     Sam James <sam@gentoo.org>
CommitDate: 2023-11-06 21:41:33 +0000

    sys-apps/portage: add 3.0.55
    
    Closes: https://bugs.gentoo.org/916256
    Closes: https://bugs.gentoo.org/915842
    Closes: https://bugs.gentoo.org/916247
    Closes: https://bugs.gentoo.org/916336
    Closes: https://bugs.gentoo.org/916601
    Closes: https://bugs.gentoo.org/916614
    Signed-off-by: Sam James <sam@gentoo.org>

 sys-apps/portage/Manifest              |   1 +
 sys-apps/portage/portage-3.0.55.ebuild | 238 +++++++++++++++++++++++++++++++++
 2 files changed, 239 insertions(+)
Comment 3 Zac Medico gentoo-dev 2024-02-03 07:21:03 UTC
I found a nice fix for this issue that's included in https://github.com/gentoo/portage/pull/1249:

commit 3efb6c538d839b78655cbfe68400cb4f839df996
Author: Zac Medico <zmedico@gentoo.org>
Date:   2024-02-02 22:32:47 -0800

    ForkProcess: Use duplicate fd_pipes in _send_fd_pipes thread
    
    In order to allow callers to manage the lifecycle of fd_pipes
    file descriptors, create duplicates for _send_fd_pipes to
    close when it has finished sending them.
    
    This fixes bug 916601 in a nice way, allowing commit
    3b1234ba69a31709cd5aec1ae070901e3a28bb7c to be reverted.
    
    Bug: https://bugs.gentoo.org/916601
    Signed-off-by: Zac Medico <zmedico@gentoo.org>
Comment 4 Larry the Git Cow gentoo-dev 2024-02-03 22:54:18 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/proj/portage.git/commit/?id=7ea1175091886baa677d11290d4b725a64e68710

commit 7ea1175091886baa677d11290d4b725a64e68710
Author:     Zac Medico <zmedico@gentoo.org>
AuthorDate: 2024-02-03 19:36:13 +0000
Commit:     Zac Medico <zmedico@gentoo.org>
CommitDate: 2024-02-03 19:55:19 +0000

    Revert "testAsyncFunctionStdin: multiprocessing spawn compat"
    
    This reverts commit 3b1234ba69a31709cd5aec1ae070901e3a28bb7c,
    since ForkProcess now solves the problem by creating temporary
    duplicate file descriptors.
    
    Bug: https://bugs.gentoo.org/916601
    Signed-off-by: Zac Medico <zmedico@gentoo.org>

 lib/portage/tests/process/test_AsyncFunction.py | 19 ++++++++-----------
 1 file changed, 8 insertions(+), 11 deletions(-)

https://gitweb.gentoo.org/proj/portage.git/commit/?id=9f96ce5105e7bd2580ae9acc34d6ebad914dae47

commit 9f96ce5105e7bd2580ae9acc34d6ebad914dae47
Author:     Zac Medico <zmedico@gentoo.org>
AuthorDate: 2024-02-03 19:36:05 +0000
Commit:     Zac Medico <zmedico@gentoo.org>
CommitDate: 2024-02-03 19:55:19 +0000

    ForkProcess: Use duplicate fd_pipes in _send_fd_pipes thread
    
    In order to allow callers to manage the lifecycle of fd_pipes
    file descriptors, create duplicates for _send_fd_pipes to
    close when it has finished sending them.
    
    This fixes bug 916601 in a nice way, allowing commit
    3b1234ba69a31709cd5aec1ae070901e3a28bb7c to be reverted.
    
    Bug: https://bugs.gentoo.org/916601
    Signed-off-by: Zac Medico <zmedico@gentoo.org>

 lib/portage/util/_async/ForkProcess.py | 55 +++++++++++++++++++++++++++++-----
 1 file changed, 48 insertions(+), 7 deletions(-)