Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 657360 - sys-apps/portage-2.3.40: emerge -fp freezes on fetch restricted ebuilds
Summary: sys-apps/portage-2.3.40: emerge -fp freezes on fetch restricted ebuilds
Status: RESOLVED FIXED
Alias: None
Product: Portage Development
Classification: Unclassified
Component: Core - Interface (emerge) (show other bugs)
Hardware: AMD64 Linux
: Normal critical (vote)
Assignee: Portage team
URL:
Whiteboard:
Keywords: InVCS, REGRESSION
Depends on:
Blocks: 651804 654390
  Show dependency tree
 
Reported: 2018-06-04 19:35 UTC by Florian Gamböck
Modified: 2018-06-06 16:33 UTC (History)
0 users

See Also:
Package list:
Runtime testing required: ---


Attachments
emerge --info (file_657360.txt,22.16 KB, text/plain)
2018-06-04 19:36 UTC, Florian Gamböck
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Florian Gamböck 2018-06-04 19:35:22 UTC
When pretending to fetch package sources (for example to display URIs for later or external downloading) for restricted packages like =www-client/google-chrome-67.0.3396.62 (RESTRICT="bindist mirror strip", culprit seems to be "mirror" here), emerge freezes and won't exit, not even after SIGINT.

Reproducible: Always

Steps to Reproduce:
1. emerge --ignore-default-opts -1fp google-chrome
Actual Results:  
These are the packages that would be fetched, in order:

Calculating dependencies... done!

https://dl.google.com/linux/chrome/deb/pool/main/g/google-chrome-stable/google-chrome-stable_67.0.3396.62-1_amd64.deb
Exception in callback AsynchronousTask.wait()        
handle: <Handle AsynchronousTask.wait()>             
Traceback (most recent call last):                   
  File "/usr/lib64/python3.5/asyncio/events.py", line 127, in _run
    self._callback(*self._args)                      
  File "/usr/lib64/python3.5/site-packages/_emerge/AsynchronousTask.py", line 84, in wait
    self._wait_hook()                                
  File "/usr/lib64/python3.5/site-packages/_emerge/AsynchronousTask.py", line 195, in _wait_hook
    self._exit_listener_stack.pop()(self)            
  File "/usr/lib64/python3.5/site-packages/_emerge/EbuildBuild.py", line 107, in _start_with_metadata
    self._prefetch_exit(prefetcher)                  
  File "/usr/lib64/python3.5/site-packages/_emerge/EbuildBuild.py", line 144, in _prefetch_exit
    retval = fetcher.execute()                       
  File "/usr/lib64/python3.5/site-packages/_emerge/EbuildFetchonly.py", line 26, in execute
    mydbapi=portdb, tree="porttree")                 
  File "/usr/lib64/python3.5/site-packages/portage/package/ebuild/doebuild.py", line 1083, in doebuild
    fd_pipes=fd_pipes)
  File "/usr/lib64/python3.5/site-packages/portage/package/ebuild/_spawn_nofetch.py", line 125, in spawn_nofetch
    return nofetch.wait()                            
  File "/usr/lib64/python3.5/site-packages/_emerge/AsynchronousTask.py", line 82, in wait
    raise asyncio.InvalidStateError('Result is not ready.')
asyncio.futures.InvalidStateError: Result is not ready.
 * Please wait 24 hours and sync your tree before reporting a bug for google-chrome fetch failures.
^C

Exiting on signal 2
^C^C^C

# Here it is completely frozen, I have to kill -9 it. By the way, the last log line is produced by pkg_nofetch of chrome ebuild, if that matters. The first SIGINT is issued after a longish period of time, not immediately or while emerge was still computing.

Expected Results:  
These are the packages that would be fetched, in order:

Calculating dependencies... done!

https://dl.google.com/linux/chrome/deb/pool/main/g/google-chrome-stable/google-chrome-stable_67.0.3396.62-1_amd64.deb

$ echo $?
0

# And not be frozen, of course.

This problem first occured today after I updated to the now stable 2.3.40.

A git bisect gave me the following:

28d3c9ed46b16757626b292dd6d7a6cae054b669 is the first bad commit
commit 28d3c9ed46b16757626b292dd6d7a6cae054b669
Author: Zac Medico <zmedico@gentoo.org>
Date:   Thu May 24 12:40:10 2018 -0700

    doebuild: eliminate redundant aux_get calls (bug 656394)

    These redundant aux_get calls triggered event loop recursion
    when called via EbuildFetchonly as reported in bug 656394.

    Bug: https://bugs.gentoo.org/656394

Exact commands that lead to this result, PWD being git-root of portage repo:
git bisect start portage-2.3.40 portage-2.3.24
git bisect run bash -c '(PATH="bin:$PATH" PYTHONPATH=pym emerge --ignore-default-opts -1fp google-chrome &>/dev/null) & pid=$!; x=; for _ in {1..10}; do [ -d /proc/$pid ] || { x=1; break; }; sleep 1; done; if [ ! "$x" ]; then kill -9 $pid; exit 1; else exit 0; fi'

With 2.3.40 being a "bad" commit and 2.3.24 being a version that worked. The run script will start the emerge in a subshell and forcibly kill it if it does not return after 10 seconds. On my machine, the working example merely took three or so seconds, maybe you have to adjust it if your machine is slower.
Comment 1 Florian Gamböck 2018-06-04 19:36:46 UTC
Created attachment 534866 [details]
emerge --info
Comment 2 Larry the Git Cow gentoo-dev 2018-06-04 23:46:56 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/proj/portage.git/commit/?id=345256c2d439c5ab580e4226f227db2819883d40

commit 345256c2d439c5ab580e4226f227db2819883d40
Author:     Zac Medico <zmedico@gentoo.org>
AuthorDate: 2018-06-04 23:32:46 +0000
Commit:     Zac Medico <zmedico@gentoo.org>
CommitDate: 2018-06-04 23:44:15 +0000

    emerge -pf: spawn pkg_nofetch asynchronously (bug 657360)
    
    For pretend mode, fix doebuild to skip the spawn_nofetch call
    that would trigger event loop recursion, and spawn pkg_nofetch
    asynchronously.
    
    Bug: https://bugs.gentoo.org/657360

 pym/_emerge/EbuildBuild.py             | 16 ++++++++++++++--
 pym/_emerge/EbuildFetchonly.py         |  5 ++++-
 pym/portage/package/ebuild/doebuild.py | 15 +++++++--------
 3 files changed, 25 insertions(+), 11 deletions(-)
Comment 3 Larry the Git Cow gentoo-dev 2018-06-05 00:35:43 UTC
The bug has been closed via the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=1b6f752ced83566176c103289159520091e63b5d

commit 1b6f752ced83566176c103289159520091e63b5d
Author:     Zac Medico <zmedico@gentoo.org>
AuthorDate: 2018-06-05 00:31:24 +0000
Commit:     Zac Medico <zmedico@gentoo.org>
CommitDate: 2018-06-05 00:35:36 +0000

    sys-apps/portage: fix emerge -pf pkg_nofetch (bug 657360)
    
    See: https://gitweb.gentoo.org/proj/portage.git/commit/?id=345256c2d439c5ab580e4226f227db2819883d40
    Closes: https://bugs.gentoo.org/657360
    Package-Manager: Portage-2.3.40, Repoman-2.3.9

 sys-apps/portage/Manifest              | 1 +
 sys-apps/portage/portage-2.3.40.ebuild | 7 +++++--
 2 files changed, 6 insertions(+), 2 deletions(-)
Comment 4 Larry the Git Cow gentoo-dev 2018-06-06 16:33:26 UTC
The bug has been closed via the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=cfe44e894ae844ab8768dfd5584506750d0be297

commit cfe44e894ae844ab8768dfd5584506750d0be297
Author:     Zac Medico <zmedico@gentoo.org>
AuthorDate: 2018-06-06 16:26:12 +0000
Commit:     Zac Medico <zmedico@gentoo.org>
CommitDate: 2018-06-06 16:27:21 +0000

    sys-apps/portage: 2.3.40-r1 revbump for regression fixes
    
    Fixes: https://bugs.gentoo.org/656610
    Fixes: https://bugs.gentoo.org/656750
    Fixes: https://bugs.gentoo.org/656942
    Fixes: https://bugs.gentoo.org/657360
    Fixes: https://bugs.gentoo.org/657420
    Fixes: https://bugs.gentoo.org/657436
    Package-Manager: Portage-2.3.40, Repoman-2.3.9

 sys-apps/portage/{portage-2.3.40.ebuild => portage-2.3.40-r1.ebuild} | 0
 1 file changed, 0 insertions(+), 0 deletions(-)