Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 120739 - Kernel source emerges extremely slow: Suggested fix.
Summary: Kernel source emerges extremely slow: Suggested fix.
Status: RESOLVED FIXED
Alias: None
Product: Portage Development
Classification: Unclassified
Component: Enhancement/Feature Requests (show other bugs)
Hardware: All Linux
: High enhancement (vote)
Assignee: Portage team
URL: http://forums.gentoo.org/viewtopic.ph...
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2006-01-28 09:09 UTC by Antony Mee
Modified: 2006-07-17 10:37 UTC (History)
2 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Antony Mee 2006-01-28 09:09:58 UTC
The attached URL is to a brief discussion on the forums (topic 3066026) about
the time it takes for an emerge of a set of kernel sources.

I suggest that this extremely long time could be cut to less that 10% of the current time taken by more efficient use of the file system and the precalculation of md5sums for tarball contents for source (uncompiled) packages. (please see the forum discussion)

Sadly for me Python is still the one that got away so I have not implemented such a fix but imagine it could be easily handled by the kernel-2.eclass.
Comment 1 Brian Harring (RETIRED) gentoo-dev 2006-01-28 11:49:10 UTC
Usually summarizing on the bug rather then pointing to the forums is wise (folks may not have a browser, or may be lazy and unwilling to dig through a potential huge thread)... ;)

Meanwhile, moving the install image directly to /usr/src isn't likely to happen.  Reason is that it's a direct modification of the livefs; what happens if the ebuild bails out after installing those files (whether in preinst/postinst, or just a dumb ass mistake in install) ?

Orphaned files- portage no longer knows about those files.  Kind of icky.
Couple of optimization points for merging; right now, if you have FEATURES="buildpkg" on, portage does some stupid stuff- install to ${D}, tarball it up, unpack the binpkg to another ${D}, then merge from that ${D} to the livefs.

Eliminating that redundant crap will knock a pretty large chunk of IO off.

Converting over to mv entirely instead of cp- can get an optimization gain out of that one, although treelink logic is going to need a serious overhaul for it (and the gain is minimal if it's crossing partitions).
Comment 2 Jakub Moc (RETIRED) gentoo-dev 2006-07-17 06:11:34 UTC
(In reply to comment #1)
> Couple of optimization points for merging; right now, if you have
> FEATURES="buildpkg" on, portage does some stupid stuff- install to ${D},
> tarball it up, unpack the binpkg to another ${D}, then merge from that ${D} to
> the livefs.

The above is already fixed in recent versions. Also, there's RESTRICT="binchecks strip" now to skip all the scanelf stuff that's redundant here. Don't think there's anything left to do here. Orphaning files is not really an option.
Comment 3 Zac Medico gentoo-dev 2006-07-17 10:37:46 UTC
Yeah, bug 138105 is fixed in >=2.1.1_pre1-r5 and the RESTRICT="binchecks" is in >=2.1.  I'm not sure if anything else needs optimizing so I think we can mark this as fixed.