Created attachment 444306 [details, diff] locale-gen parallelism patch Currently, locale-gen clamps parallelism to 1, since it generates a locale archive (one file) instead of individual locales. The tool used (localedef) does no locking on manipulating the locale-archive file, so parallelism ends up (at best) generating one locale in the file. More often, parallelism just garbles the whole thing. The attached patch separates locale generation from addition to the archive by doing the generation step first (in parallel) and then adding the generated locales to the archive in one process and then removing the individual files. On a not-too-shabby 4-CPU Alpha (the arch teams dev/build machine), this dramatically reduces the time to generate all locales (when doing weekly stage builds, for example): $ time ./locale-gen -A -j1 -d out/ >/dev/null real 42m41.692s user 41m6.520s sys 1m28.443s $ rm out/usr/lib/locale/locale-archive $ time ./locale-gen -A -j4 -d out/ >/dev/null real 14m26.445s user 42m8.829s sys 1m34.122s I'm sure other small arches would welcome this speed increase, too. And even on an i7-2600K, the speedup is not trivial: $ time ./locale-gen -A -j1 -d out/ >/dev/null real 3m50.603s user 3m37.950s sys 0m10.708s $ rm out/usr/lib64/locale/locale-archive $ time ./locale-gen -A -j4 -d out/ >/dev/null real 1m16.758s user 3m54.573s sys 0m10.729s $ rm out/usr/lib64/locale/locale-archive $ time ./locale-gen -A -j8 -d out/ >/dev/null real 1m2.680s user 5m41.405s sys 0m14.335s
Updated in glibc's gentoo patchset as: https://sources.gentoo.org/cgi-bin/viewvc.cgi/gentoo/src/patchsets/glibc/extra/locale/locale-gen?view=diff&r1=text&tr1=1.42&r2=text&tr2=1.41&diff_format=h
Pushed to 2.24-r4 and 2.25-r3 as: https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=280e225b50f7ab21e84d112087784c5424ba00be Thank you Tobias!
