Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 747016 - sys-apps/portage: allow configurable lru_cache scaling
Summary: sys-apps/portage: allow configurable lru_cache scaling
Alias: None
Product: Portage Development
Classification: Unclassified
Component: Core (show other bugs)
Hardware: All Linux
: Normal normal with 1 vote (vote)
Assignee: Portage team
Depends on:
Blocks: 835380
  Show dependency tree
Reported: 2020-10-07 08:36 UTC by Jason A. Donenfeld
Modified: 2022-07-27 03:28 UTC (History)
4 users (show)

See Also:
Package list:
Runtime testing required: ---


Note You need to log in before you can comment on or make changes to this bug.
Description Jason A. Donenfeld gentoo-dev 2020-10-07 08:36:12 UTC
The recent addition of these lines resulted in a big performance boost:

20:from functools import lru_cache
13:from functools import lru_cache

The values 1024 and 10240 were chosen for some trade-off of speed and ram usage. However, with 64 gigs of ram, my trade-offs might be different than your trade-offs. Therefore, I propose a user-configurable scaling knob, PORTAGE_LRU_CACHE_SCALING_FACTOR. These would then be used like:


This way, users with excess ram could trade it for a performance boost.

Reproducible: Always
Comment 1 Zac Medico gentoo-dev 2020-10-10 06:38:26 UTC
We can hook this into the function which is used to initialize things with user configuration settings.
Comment 2 Zac Medico gentoo-dev 2020-12-21 00:18:43 UTC
In there's a rejected proposal to allow runtime resize of lru_cache instances, with rationale give that a lru_cache instance has a __wrapped__ attribute which can be used to unwrap and re-wrap the wrapped function.

A disadvantage of the function re-wrapping approach is that it would not be possible to replace all previous imports of the wrapped function with new lru_cache instance, so we would need to add an additional decorator to serve as a layer of indirection for cases when the wrapped function has been imported into another module. However, I expect the performance impact of this extra decorator layer to be negligible.