Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 436084 - emerge performance bad due to unnecessary binary package scanning
Summary: emerge performance bad due to unnecessary binary package scanning
Alias: None
Product: Portage Development
Classification: Unclassified
Component: Core - Interface (emerge) (show other bugs)
Hardware: All Linux
: Normal enhancement (vote)
Assignee: Portage team
Keywords: InVCS
Depends on:
Blocks: 835380 431026
  Show dependency tree
Reported: 2012-09-24 10:08 UTC by Konstantin Münning
Modified: 2023-05-23 12:43 UTC (History)
0 users

See Also:
Package list:
Runtime testing required: ---


Note You need to log in before you can comment on or make changes to this bug.
Description Konstantin Münning 2012-09-24 10:08:20 UTC

Quite a while now I am seeing a continuously decreasing performance of emerge. One of the worst performance killers seems to be that emerge almost always scans binary packages in PKGDIR even if no binary packages are requested (no -k|-K|-g|-G flag). This includes operations which do not need binary packages at all like emerge --depclean.

If there is no important reason to keep current behavior it would be great to make emerge scan PKGDIR only when binary packages are requested.
Comment 2 Zac Medico gentoo-dev 2012-09-26 04:37:43 UTC
This is fixed in and 2.2.0_alpha131.
Comment 3 Konstantin Münning 2012-09-26 14:40:15 UTC
That's great. Tested and now an emerge -pv portage takes 5 minutes instead of 14, so a huge improvement.

Are there any further plans to improve performance? After package scanning a worthwhile optimization target seem to be dependency calculations. If there is anything I could do, let me know but I have no experience with python profiling.
Comment 4 Zac Medico gentoo-dev 2012-09-26 16:17:33 UTC
For python profiles, I have a couple of scripts that you can use to get started:

Usage is like this: emerge.pstats emerge <args> emerge.pstats > emerge.pstats.txt

The second command converts the profiling stats to a human-readable form, which is easy to analyze.