Summary: | Possible memory leak in portage. | ||
---|---|---|---|
Product: | Portage Development | Reporter: | Aaron Kulbe (RETIRED) <superlag> |
Component: | Core - Interface (emerge) | Assignee: | Portage team <dev-portage> |
Status: | RESOLVED INVALID | ||
Severity: | normal | CC: | hppa, ppc, sparc |
Priority: | High | ||
Version: | 2.1 | ||
Hardware: | AMD64 | ||
OS: | Other | ||
Whiteboard: | |||
Package list: | Runtime testing required: | --- |
Description
Aaron Kulbe (RETIRED)
2006-02-26 22:59:28 UTC
Is it reproducable? Technically I want to say "Portage doesn't have memory leaks" because it would be an error in Python's garbage collection. It is my understanding that emerge --sync would end, killing one python interpreter, linux should reclaim that memory to the heap and it would start over for the emerge -au world. You also mention that memory usage is 514MB even after emerge and rsync exited...This is after emerge -au world finished? What process was the RAM given to then? ps -auxfw should give a comprehensive list. Was perhaps "python" still running? (In reply to comment #1) > Is it reproducable? Technically I want to say "Portage doesn't have memory > leaks" because it would be an error in Python's garbage collection. Certain obscene uses will trip up gc actually, leading to python holding onto things for a bit longer then you'd expect. Either way, that statement isn't accurate- gc is effectively smart ptrs (ref counting) with periodic cycle detection. So... if we have refs to objects left, garbage collection can't do anything about it. > It is my > understanding that emerge --sync would end, killing one python interpreter, > linux should reclaim that memory to the heap and it would start over for the > emerge -au world. He stated first emerge instance used 155, second used half a gig (eg, his statement is accurate if it's leaking). The *rough* emerge mem usage for 32bit arches is ~20 mb for emerge --metadata runs; for 64bit, would expect it to be in the range of 25-30 (sizeof(ptr)), so if you're seeing 155 something is weird. Custom backend? A guess offhand is that this is either 1) cache leakage (the 5-7x increase I don't see in --metadata runs though) 2) config instance leakage. This one strikes me as probable, since repoman has repeatedly exposed config leakage/bloat (occasion versions of repoman in the past would clear 2 gb for a full tree run). Reading your comments on the bug, I think I need to clarify something. The figures 155MB before syncing, and 514MB after syncing were totals. Totals for ALL processes. I used top to tell. I wasn't looking at memory usage for single process. When I ran the sync, I watched the total memory usage climb rapidly, but never return to original levels after 'emerge sync && emerge -au world' was finished. And... a full 180 from me. :) If memory isn't being used for anything (eg, not used by an app or by the kernel data structures) linux uses the free memory to buffer data from disk- disk access is slow, the memory isn't (currently) used for anything so it uses it for buffers. It'll release the memory to any app that requests memory, but the unused memory it uses for io buffering; after a sync, and after a emerge -Du world portage has accessed a _lot_ of files, thus if you've got the memory, a lot will be buffered. crap... wrong resolution. there we go... |