Summary: | installing binpkgs in parallel doesn't actually install in parallel | ||
---|---|---|---|
Product: | Portage Development | Reporter: | SpanKY <vapier> |
Component: | Binary packages support | Assignee: | Portage team <dev-portage> |
Status: | CONFIRMED --- | ||
Severity: | normal | CC: | gentoo, jrosenth, sam, zmedico |
Priority: | Normal | ||
Version: | unspecified | ||
Hardware: | All | ||
OS: | Linux | ||
See Also: | https://bugs.gentoo.org/show_bug.cgi?id=890907 | ||
Whiteboard: | |||
Package list: | Runtime testing required: | --- | |
Bug Depends on: | |||
Bug Blocks: | 835380 | ||
Attachments: | Portage Versions Bisection Findings.pdf |
Description
SpanKY
2022-08-23 00:11:22 UTC
ccing zmedico in case he has any ideas Sam asked me to update this bug with what I know. Firstly, the document go/cros-portage-bisection-findings from within Google has a bit more analysis. I'm not sure why all of it didn't find its way here. I also emailed Zac a few months ago. (In reply to SpanKY from comment #0) > it's been hard bisecting things down due to the async migrations breaking > things, but we've identified at least one place where there was a large > regression. reverting that in 3.0.21 seems to bring back a lot, but not > all, of the performance. > https://gitweb.gentoo.org/proj/portage.git/commit/ > ?id=d66e9ec0b10522528d62e18b83e012c1ec121787 This commit is in portage-2.3.90. In a private email with Zac, he notes that this commit was reverted in commit https://gitweb.gentoo.org/proj/portage.git/commit/?id=71ae5a58fe72bc32dce030210a73ea5c9eeb4a1c which is in portage-2.3.97. From looking at the Google doc, I suspect that vapier meant a different commit (this one isn't mentioned in the doc, and it was already reverted before 3.0.21) The two commits mentioned in the Google doc are 1) https://gitweb.gentoo.org/proj/portage.git/commit/?id=9b755b46f9e88f25fecada0a32095ea614a73b57 (in portage-2.3.99), reported to cause a large increase in time taken to calculate dependencies (+800%). This may have been the issue mitigated by https://gitweb.gentoo.org/proj/portage.git/commit/?id=839ab46be1777e5886da28b98b53a462b992c5bf 2. https://gitweb.gentoo.org/proj/portage.git/commit/?id=50da2e16599202b9ecb3d4494f214a0d30b073d (in portage-2.3.93), reported to keep various processes alive for longer than before, which leads to less parallelization. I believe this is the commit vapier meant to cite in the original report. Zac also noted that the rdep_cache in https://chromium-review.googlesource.com/c/chromiumos/third_party/portage_tool/+/3780786 may help here. Created attachment 864225 [details]
Portage Versions Bisection Findings.pdf
Here's the document.
|