Summary: | repoman: add a way to limit repoman to subset of profiles (similar to --include-arches= for arches) | ||
---|---|---|---|
Product: | Portage Development | Reporter: | Sergei Trofimovich (RETIRED) <slyfox> |
Component: | Repoman | Assignee: | Portage team <dev-portage> |
Status: | RESOLVED FIXED | ||
Severity: | normal | ||
Priority: | Normal | ||
Version: | unspecified | ||
Hardware: | All | ||
OS: | Linux | ||
Whiteboard: | |||
Package list: | Runtime testing required: | --- | |
Attachments: | 0001-repoman-add-include-profiles-PROFILES.patch |
Description
Sergei Trofimovich (RETIRED)
2019-11-18 08:26:26 UTC
Created attachment 596746 [details, diff]
0001-repoman-add-include-profiles-PROFILES.patch
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/proj/portage.git/commit/?id=e9bb1e9681685f4e4d7174f51751356fd3f67d0c commit e9bb1e9681685f4e4d7174f51751356fd3f67d0c Author: Sergei Trofimovich <slyfox@gentoo.org> AuthorDate: 2019-11-19 00:21:13 +0000 Commit: Zac Medico <zmedico@gentoo.org> CommitDate: 2019-11-20 10:29:14 +0000 repoman: add --include-profiles=PROFILES repoman slows down ~linearly with amount of profiles being scanned. In case of amd64 we have 28 stable profiles. To speed up processing and fit into time budged of various CIs we can split the work across different processes that handle different profiles. Example benchmark on ::haskell overlay: $ ./repoman full --include-arches=amd64 ~65 minutes $ ./repoman full --include-profiles=default/linux/amd64/17.0 ~4 minutes This allows for a crude sharding of work across processes and allows for cheap tree-wide scans for early failures. Bug: https://bugs.gentoo.org/700456 Signed-off-by: Sergei Trofimovich <slyfox@gentoo.org> Signed-off-by: Zac Medico <zmedico@gentoo.org> repoman/lib/repoman/actions.py | 4 ++++ repoman/lib/repoman/argparser.py | 7 +++++++ repoman/lib/repoman/modules/scan/depend/__init__.py | 3 ++- repoman/lib/repoman/modules/scan/depend/profile.py | 9 +++++++-- repoman/lib/repoman/scanner.py | 5 +++++ repoman/lib/repoman/tests/simple/test_simple.py | 1 + repoman/man/repoman.1 | 4 ++++ 7 files changed, 30 insertions(+), 3 deletions(-) The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=0af2015bc57a267f5f67610a73bde5da16bcc6d4 commit 0af2015bc57a267f5f67610a73bde5da16bcc6d4 Author: Zac Medico <zmedico@gentoo.org> AuthorDate: 2019-11-27 04:18:16 +0000 Commit: Zac Medico <zmedico@gentoo.org> CommitDate: 2019-11-27 04:19:23 +0000 app-portage/repoman: Bump to version 2.3.19 #667432 Rename DCO_SIGNED_OFF_BY config variable to SIGNED_OFF_BY. #700456 Add --include-profiles=PROFILES Bug: https://bugs.gentoo.org/667432 Bug: https://bugs.gentoo.org/700456 Package-Manager: Portage-2.3.80, Repoman-2.3.19 Signed-off-by: Zac Medico <zmedico@gentoo.org> app-portage/repoman/Manifest | 1 + app-portage/repoman/repoman-2.3.19.ebuild | 63 +++++++++++++++++++++++++++++++ 2 files changed, 64 insertions(+) Using per-profile charding allows me to shrink scan time down to 5 minutes per profiles: https://travis-ci.org/gentoo-haskell/gentoo-haskell/builds/651819453?utm_source=github_status&utm_medium=notification Thank you! |