Bug 570472

Summary:	sys-apps/portage: allow to depclean in parallel
Product:	Portage Development	Reporter:	Pacho Ramos <pacho>
Component:	Core	Assignee:	Portage team <dev-portage>
Status:	CONFIRMED ---
Severity:	normal	CC:	kredba, lssndrbarbieri
Priority:	Normal
Version:	unspecified
Hardware:	All
OS:	Linux
Whiteboard:
Package list:		Runtime testing required:	---
Bug Depends on:
Bug Blocks:	835380

Description Pacho Ramos gentoo-dev

2016-01-01 11:11:55 UTC

This is an issue I suffer from time to time when I run emerge --depclean to clean a lot of packages I usually end up having installed but I don't really need. For example, now I have a list of more than 700 packages to depclean.

Something that surprises me is how slow does portage clean every package. Running iotop (and looking to the HD led) I see I/O is not a bottleneck at all, on the other hand, the "emerge" process looks to be the major bottleneck as it runs only 1 process and takes around 90-100% of CPU

I was wondering then if maybe emerge --depclean could honor the "--jobs" option to allow it to run multiple "emerge" processes depcleaning different parts of the big list of packages to remove. For example in this concrete case, it would be nice if --jobs=2 would end up with running 2 emerge processes, one for the first half of the list to depclean and other for the second time.

What do you think? Well, maybe if it's too hard to make it behave in that way with the same "emerge" process, a wrapper to run multiple emerge --depclean splitting the list of packages to remove could also help :/

Thanks a lot

Comment 1 Zac Medico gentoo-dev

2016-01-02 05:31:32 UTC

We can parallelize it, but for maximum performance you'll need the equivalent of FEATURES="parallel-install -ebuild-locks". Also, disabling preserve-libs will improve performance, since preserve-libs uses a global lock when it's checking to see if any libraries need to be preserved or if any preserved libraries can be deleted.