Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 729644 - emerge seems to open and close log files for each (unmerge?) line it wants to write
Summary: emerge seems to open and close log files for each (unmerge?) line it wants to...
Status: CONFIRMED
Alias: None
Product: Portage Development
Classification: Unclassified
Component: Unclassified (show other bugs)
Hardware: All Linux
: Normal normal (vote)
Assignee: Portage team
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2020-06-25 16:27 UTC by Jeroen Roovers (RETIRED)
Modified: 2020-06-26 06:00 UTC (History)
0 users

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Jeroen Roovers (RETIRED) gentoo-dev 2020-06-25 16:27:39 UTC
Especially during unmerge/replacement of a different version, emerge appears to progress very slowly. By using an NFS-based path to store the log files, I can see that the log file is opened, written to and closed again for each line it wants to write. I suspect this is done for other phases as well, only less obviously. Even on fast hosts, unmerging a package with many files (say, a kernel) and storing the log on NFS takes exceptionally long.

I recently cleaned up old gentoo-sources installs and it took $reasonably_fast_host a couple of days(!) to unmerge some forty packages. So that's what got me here.
Comment 1 Zac Medico gentoo-dev 2020-06-26 05:52:19 UTC
It's the _display_merge method that opens the log each time that it's called:

https://gitweb.gentoo.org/proj/portage.git/tree/lib/portage/dbapi/vartree.py?h=portage-2.3.99#n2404
Comment 2 Zac Medico gentoo-dev 2020-06-26 06:00:17 UTC
We can open the file once at the beginning of the _unmerge_pkgfiles method, and pass it into all of the _display_merge calls.