Whereas portage-2.2.00.15274 (and all previous) perform correctly, with portage-2.2.00.15320, single ebuilds still work as expected. However, larger builds (I've experienced this with builds of 90 packages simultaneously and 301 packages simultaneously) will lock-up after the very first package's "checking miscfile checksums ;-) ... [ ok ]" (noting that the "ok" is printed, so presumably whatever happens next is breaking before outputting anything). When running through an strace-equivalent, it doesn't appear that portage is working in the background or waiting for a forked process to complete - python simply stops making further systems calls. 'top' shows the python process as being idle, to it doesn't appear to be doing any heavy processing either. With portage-2.2.00.15274 and below, there is no delay at all at this stage.
Created attachment 219001 [details] par (strace-equivalent) output Here's a weird thing... if I run emerge from portage-2.2.00.15320 alone or through 'par', the problem occurs without fail. If I run the *exact* same command (through par) but pipe the output to tee or even redirect it to a file, then emerge runs correctly!? Attached log shows 'emerge' running - and then simply stopping doing anything...
smells like poll/select trouble on pipe/ptys again
Also happens with sys-apps/portage-2.2.00.15335. Only seems to affect builds of large numbers of packages - with single packages (or just now, four at once) everything works correctly.
% cvs log -r15275:15320 ------------------------------------------------------------------------ r15299 | grobian | 2010-01-31 10:40:03 +0100 (Sun, 31 Jan 2010) | 17 lines Merged from trunk -r15272:15280 | 15277 | Define userpriv_groups earlier in order to avoid an | | zmedico | AttributeError from lazy import code. Thanks to antarus for | | | reporting. | | 15278 | Bug #277902 - When excluding config files, instead of using | | zmedico | an empty file as a placeholder, use a file containing a | | | comment like this: # empty file because --include-config=n | | | when `quickpkg` was used | | 15279 | Fix conditional logic for userpriv_groups intialization. | | zmedico | | | 15280 | Fix code from r15278 for python3 unicode compatibility. | | zmedico | | ------------------------------------------------------------------------ r15300 | grobian | 2010-01-31 10:41:21 +0100 (Sun, 31 Jan 2010) | 17 lines Merged from trunk -r15280:15285 | 15281 | Deprecate the first parameter of the portdbapi constructor | | zmedico | since it is always the same as mysettings['PORTDIR']. | | 15282 | Use tuples instead of lists for immutability. | | zmedico | | | 15283 | For immutable config attributes, use shallow copy for speed | | zmedico | and memory conservation. | | 15284 | For immutable config attributes, use shallow copy for speed | | zmedico | and memory conservation. | | 15285 | For immutable config attributes, use shallow copy for speed | | zmedico | and memory conservation. | ------------------------------------------------------------------------ r15301 | grobian | 2010-01-31 10:43:08 +0100 (Sun, 31 Jan 2010) | 24 lines Merged from trunk -r15285:15292 | 15286 | fix: python dbm module doesn't accept unicode as key | | volkmar | | | 15287 | Define __slots__ in all classes. | | zmedico | | | 15288 | In config._expandLicenseToken(), make sure traversed_groups | | zmedico | is only initialized when appropriate. | | 15289 | Fix concatenation of list + tuple in | | zmedico | Manifest._createManifestEntries(). | | 15290 | Don't generate DeprecationWarning for portdbapi arguments. | | zmedico | Wait until new portdbapi constructor is in stable portage | | | first. | | 15291 | fix flat_list cache backend looks like it was outdated wrt. | | volkmar | template cache backend | | 15292 | Show a DeprecationWarning if the deprecated pordbapi | | zmedico | constructor parameter is used incorrectly. | ------------------------------------------------------------------------ r15302 | grobian | 2010-01-31 10:44:08 +0100 (Sun, 31 Jan 2010) | 25 lines Merged from trunk -r15292:15298 | 15293 | Fix: change os.mkdir to os.makedirs in case of there are | | volkmar | more than one directory depth missing. According to | | | docs.python.org, permissions may be not respected with | | | os.makedirs... with some systems. | | 15294 | Avoid NameError when cwd is $FILESDIR. Thanks to Christian | | zmedico | Ruppert <idl0r@g.o> for reporting. | | 15295 | Bug #302764 - Inside __iter__, only recurse 1 deep, in order | | zmedico | to avoid iteration over entries from another nested cache | | | instance. This can happen if the user nests an overlay | | | inside /usr/portage/local. Thanks to Vlastimil Babka | | | <caster@g.o> for this patch. | | 15296 | Make __iter__ use list.pop() instead of pop(0), for greater | | zmedico | efficiency. | | 15297 | In __iter__, handle OSError from lstat in case a cache entry | | zmedico | disappears. | | 15298 | Reverting r15293, it was actually not needed. | | volkmar | | ------------------------------------------------------------------------ r15319 | grobian | 2010-02-04 20:03:41 +0100 (Thu, 04 Feb 2010) | 1 line fix backtrace with -r0 revisions, by Paul Varner, bug #303145 ------------------------------------------------------------------------ r15320 | grobian | 2010-02-04 20:09:17 +0100 (Thu, 04 Feb 2010) | 49 lines Merged from trunk -r15303:15318 | 15304 | Bug #302937 - Handle declare -r without assignment. | | zmedico | | | 15305 | Validate categories. | | zmedico | | | 15306 | Deprecate pordbapi.mysettings since the portdbapi.settings | | zmedico | alias is now supported by stable portage. | | 15307 | Bug #298141 - Make /etc/portage/sets relative to | | zmedico | PORTAGE_CONFIGROOT. Thanks to Martin Gysel (bearsh) | | | <m.gysel@gmx.ch> for this patch. | | 15308 | Fix @module-rebuild to support $ROOT. | | zmedico | | | 15309 | Clean up config incrementals handling. | | zmedico | | | 15310 | Simplify the conditional that triggers calculation of A and | | zmedico | AA variables. | | 15311 | Deallocate config instance when necessary, to avoid memory | | zmedico | leak when in prefetch mode. | | 15312 | Optimize parallel-fetch for the case where all files are | | zmedico | already fetched and have the correct size. In this case we | | | can avoid the expense of spawning ebuild(1). | | 15313 | Make sure the fetcher process correctly inherits | | zmedico | PORTAGE_CONFIGROOT. | | 15314 | Fix deprecated portdbapi.mysettings reference. | | zmedico | | | 15315 | Use stat rather than lstat since portage.fetch() creates | | zmedico | symlinks when PORTAGE_RO_DISTDIRS is used. | | 15316 | Use Package.use.enabled where appropriate. | | zmedico | | | 15317 | Bug #295197 - Output a newline after valid user input is | | zmedico | received. | | 15318 | Make verify_all() do checksums in sorted order by hash name. | | zmedico | | ------------------------------------------------------------------------
I think only r15312 sounds possibly suspicious.
r15312 would seem to match the conditions where this has been observed...
Created attachment 221345 [details, diff] r15312 can you reverse-apply this patch to verify if this is the one causing trouble for you?
