Summary: | sys-apps/portage-2.2_rc1 emerge/ebuild gets into a tight loop when merging glibc-2.8_p20080602 | ||
---|---|---|---|
Product: | Portage Development | Reporter: | Will Saxon <saxonww> |
Component: | Core | Assignee: | Portage team <dev-portage> |
Status: | RESOLVED FIXED | ||
Severity: | normal | CC: | ari, teidakankan |
Priority: | High | Keywords: | InVCS, REGRESSION |
Version: | unspecified | ||
Hardware: | AMD64 | ||
OS: | Linux | ||
See Also: | https://bugs.gentoo.org/show_bug.cgi?id=610702 | ||
Whiteboard: | |||
Package list: | Runtime testing required: | --- | |
Bug Depends on: | |||
Bug Blocks: | 835380, 210077 | ||
Attachments: |
prevent infinite loop in dblink._preserve_libs()
cache results to improve performance |
Description
Will Saxon
2008-06-22 23:01:56 UTC
I am not able to get an attachment uploaded due to a bugzilla internal error. Here is an excerpt of the attachment, which is just a capture of strace running against the ebuild process: lstat("/usr/kde/3.5/lib64", {st_mode=S_IFDIR|0755, st_size=20480, ...}) = 0 lstat("/usr", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0 lstat("/usr/kde", {st_mode=S_IFDIR|0755, st_size=16, ...}) = 0 lstat("/usr/kde/3.5", {st_mode=S_IFDIR|0755, st_size=111, ...}) = 0 lstat("/usr/kde/3.5/lib64", {st_mode=S_IFDIR|0755, st_size=20480, ...}) = 0 lstat("/usr", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0 lstat("/usr/kde", {st_mode=S_IFDIR|0755, st_size=16, ...}) = 0 lstat("/usr/kde/3.5", {st_mode=S_IFDIR|0755, st_size=111, ...}) = 0 lstat("/usr/kde/3.5/lib64", {st_mode=S_IFDIR|0755, st_size=20480, ...}) = 0 lstat("/usr", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0 lstat("/usr/kde", {st_mode=S_IFDIR|0755, st_size=16, ...}) = 0 lstat("/usr/kde/3.5", {st_mode=S_IFDIR|0755, st_size=111, ...}) = 0 lstat("/usr/kde/3.5/lib64", {st_mode=S_IFDIR|0755, st_size=20480, ...}) = 0 lstat("/usr", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0 lstat("/usr/lib64", {st_mode=S_IFDIR|0755, st_size=53248, ...}) = 0 lstat("/usr", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0 lstat("/usr/kde", {st_mode=S_IFDIR|0755, st_size=16, ...}) = 0 lstat("/usr/kde/3.5", {st_mode=S_IFDIR|0755, st_size=111, ...}) = 0 lstat("/usr/kde/3.5/lib64", {st_mode=S_IFDIR|0755, st_size=20480, ...}) = 0 lstat("/lib64", {st_mode=S_IFDIR|0755, st_size=8192, ...}) = 0 lstat("/lib64", {st_mode=S_IFDIR|0755, st_size=8192, ...}) = 0 lstat("/lib64", {st_mode=S_IFDIR|0755, st_size=8192, ...}) = 0 lstat("/lib64", {st_mode=S_IFDIR|0755, st_size=8192, ...}) = 0 lstat("/usr", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0 lstat("/usr/lib64", {st_mode=S_IFDIR|0755, st_size=53248, ...}) = 0 lstat("/usr", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0 lstat("/usr/lib64", {st_mode=S_IFDIR|0755, st_size=53248, ...}) = 0 lstat("/usr", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0 lstat("/usr/kde", {st_mode=S_IFDIR|0755, st_size=16, ...}) = 0 lstat("/usr/kde/3.5", {st_mode=S_IFDIR|0755, st_size=111, ...}) = 0 lstat("/usr/kde/3.5/lib64", {st_mode=S_IFDIR|0755, st_size=20480, ...}) = 0 lstat("/usr", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0 lstat("/usr/qt", {st_mode=S_IFDIR|0755, st_size=14, ...}) = 0 lstat("/usr/qt/3", {st_mode=S_IFDIR|0755, st_size=151, ...}) = 0 lstat("/usr/qt/3/lib64", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0 lstat("/usr", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0 lstat("/usr/lib64", {st_mode=S_IFDIR|0755, st_size=53248, ...}) = 0 lstat("/usr", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0 lstat("/usr/lib64", {st_mode=S_IFDIR|0755, st_size=53248, ...}) = 0 lstat("/usr", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0 lstat("/usr/lib64", {st_mode=S_IFDIR|0755, st_size=53248, ...}) = 0 lstat("/usr", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0 lstat("/usr/lib64", {st_mode=S_IFDIR|0755, st_size=53248, ...}) = 0 lstat("/usr", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0 lstat("/usr/lib64", {st_mode=S_IFDIR|0755, st_size=53248, ...}) = 0 lstat("/usr", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0 lstat("/usr/lib64", {st_mode=S_IFDIR|0755, st_size=53248, ...}) = 0 lstat("/usr", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0 lstat("/usr/lib64", <unfinished ...> Process 1899 detached The attachment I have is ~3MB uncompressed and it is just more of the above. Created attachment 158059 [details, diff]
prevent infinite loop in dblink._preserve_libs()
I haven't tried to reproduce the bug, but I think this might fix it.
(In reply to comment #2) > Created an attachment (id=158059) [edit] > prevent infinite loop in dblink._preserve_libs() > > I haven't tried to reproduce the bug, but I think this might fix it. > Tried this on ~x86. Patch applied cleanly, but still hung at same point addressed above. Downgraded to portage-2.1.5.6 and glibc finished merge successfully. Created attachment 158093 [details, diff]
cache results to improve performance
The problem is actually a performance issue (at least it is with the first patch applied). It's most noticeable with glibc since it provides so many libs.
This patch greatly improves the performance of dblink._preserve_libs() calls by caching results of LinkageMap.findProviders(), LinkageMap.findConsumers(), and os.path.realpath() calls .
(In reply to comment #4) > Created an attachment (id=158093) [edit] > cache results to improve performance > > The problem is actually a performance issue (at least it is with the first > patch applied). It's most noticeable with glibc since it provides so many libs. > > This patch greatly improves the performance of dblink._preserve_libs() calls by > caching results of LinkageMap.findProviders(), LinkageMap.findConsumers(), and > os.path.realpath() calls . > This patch worked fine here and the actual file install process was a lot quicker then portage-2.1.5.6 once it began -- which seems to take a long time on glibc. Thanks for your help. The performance issue seems major. I see same on a fast quad (2008.0, portage 2.2-rc1, gcc-4.2.4. ~amd64). I did wait 5 minutes.. The patch should be applied asap. This is fixed in 2.2_rc2. |