The check_reverse_dependencies function significantly benefits from lru_cache: https://archives.gentoo.org/gentoo-portage-dev/message/621e50daab7ac041cc9fc7e9debf2f5e https://archives.gentoo.org/gentoo-portage-dev/message/bdcf3c2d85fc5267c58ed2c3fa7a6128
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/proj/portage.git/commit/?id=839ab46be1777e5886da28b98b53a462b992c5bf commit 839ab46be1777e5886da28b98b53a462b992c5bf Author: Pin-yen Lin <treapking@chromium.org> AuthorDate: 2022-11-25 03:36:27 +0000 Commit: Zac Medico <zmedico@gentoo.org> CommitDate: 2022-11-28 00:37:41 +0000 Add caching to _slot_operator_check_reverse_dependencies Add lru_cache to speed up the running time of "Calculating dependencies". In a ChromeOS use case, this patch decreases the running time from 311s to 197s with almost no memory usage increase. Bug: https://bugs.gentoo.org/883071 Signed-off-by: Pin-yen Lin <treapking@chromium.org> Signed-off-by: Zac Medico <zmedico@gentoo.org> lib/_emerge/depgraph.py | 1 + 1 file changed, 1 insertion(+)
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/proj/portage.git/commit/?id=d0dd4918fae61f206925a7d013f49fd84a4d3444 commit d0dd4918fae61f206925a7d013f49fd84a4d3444 Author: Sam James <sam@gentoo.org> AuthorDate: 2022-11-30 01:21:45 +0000 Commit: Sam James <sam@gentoo.org> CommitDate: 2022-11-30 01:22:35 +0000 NEWS: update Bug: https://bugs.gentoo.org/883071 Signed-off-by: Sam James <sam@gentoo.org> NEWS | 4 ++++ 1 file changed, 4 insertions(+)
The bug has been closed via the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=457170222d9a1133406b90955269580e4db2ee2b commit 457170222d9a1133406b90955269580e4db2ee2b Author: Sam James <sam@gentoo.org> AuthorDate: 2022-12-01 05:16:52 +0000 Commit: Sam James <sam@gentoo.org> CommitDate: 2022-12-01 05:16:52 +0000 sys-apps/portage: add 3.0.40 Closes: https://bugs.gentoo.org/814380 Closes: https://bugs.gentoo.org/850127 Closes: https://bugs.gentoo.org/881383 Closes: https://bugs.gentoo.org/883071 Closes: https://bugs.gentoo.org/883307 Closes: https://bugs.gentoo.org/883437 Signed-off-by: Sam James <sam@gentoo.org> sys-apps/portage/Manifest | 1 + sys-apps/portage/portage-3.0.40.ebuild | 283 +++++++++++++++++++++++++++++++++ 2 files changed, 284 insertions(+)
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/proj/portage.git/commit/?id=ca2728e9bc57b749408cf6c2a2766dab93595ca0 commit ca2728e9bc57b749408cf6c2a2766dab93595ca0 Author: Sam James <sam@gentoo.org> AuthorDate: 2023-09-21 15:47:04 +0000 Commit: Sam James <sam@gentoo.org> CommitDate: 2023-09-21 15:47:04 +0000 NEWS: update Bug: https://bugs.gentoo.org/883071 Signed-off-by: Sam James <sam@gentoo.org> NEWS | 2 ++ 1 file changed, 2 insertions(+) https://gitweb.gentoo.org/proj/portage.git/commit/?id=53236c55ba8c8106afc80fa113a876ae5784f1ed commit 53236c55ba8c8106afc80fa113a876ae5784f1ed Author: Daniel Harding <dharding@living180.net> AuthorDate: 2023-09-21 07:42:50 +0000 Commit: Sam James <sam@gentoo.org> CommitDate: 2023-09-21 15:46:04 +0000 depgraph: increase reverse dep cache size To avoid blowing out the cache and destroying performance, increase the maximum size of the memoization cache for the depgraph _slot_opererator_check_reverse_dependencies() method from 100 to 1000. Bug: https://bugs.gentoo.org/883071 Signed-off-by: Daniel Harding <dharding@living180.net> Closes: https://github.com/gentoo/portage/pull/1092 Signed-off-by: Sam James <sam@gentoo.org> lib/_emerge/depgraph.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) https://gitweb.gentoo.org/proj/portage.git/commit/?id=c6ee4348df65210ee4c052b894cbc9f4f54f6ce8 commit c6ee4348df65210ee4c052b894cbc9f4f54f6ce8 Author: Daniel Harding <dharding@living180.net> AuthorDate: 2023-09-10 06:51:13 +0000 Commit: Sam James <sam@gentoo.org> CommitDate: 2023-09-21 15:46:00 +0000 depgraph: improve reverse dep caching Avoid applying @functools.lru_cache() to the depgraph._slot_operator_check_reverse_depencencies() method, as applying the lru_cache decorator to a method can prevent the class instances from being garbage collected due to self being stored in the cache as part of the args key[1]. Instead, set up a per-instance cache in the __init__() method. [1] https://stackoverflow.com/q/33672412/3573385 Bug: https://bugs.gentoo.org/883071 Signed-off-by: Daniel Harding <dharding@living180.net> Signed-off-by: Sam James <sam@gentoo.org> lib/_emerge/depgraph.py | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-)
