The multiprocessing.eclass was committed in 2012 to accommodate running parallel jobs in bash (which was a bad idea in the first place). The eclass code is complex, needed multiple fixes over time and a backwards bash compatibility branch. It sustains a high bus factor -- I doubt any Gentoo developer will be willing or capable of maintaining it once vapier leaves. At this precise moment (i.e. ~6.5yr after committing the eclass), it is used by 5 packages in total: dev-lang/jimtcl, media-tv/kodi, net-misc/suite3270, sys-apps/sandbox and sys-libs/ncurses. I've verified that for all 5 packages, the multijob_* code has been added by vapier. I think this quite reasonably proves that the feature is not considered helpful by any other developer. In all of the cases except for media-tv/kodi, the multijob_* logic is used to run configure scripts in parallel. In the fifth case, it's used to run autoreconf in parallel. All of those cases qualify for minor speed gain, at the cost of unreadable output and potential problems. I would like to point out that all those uses actually violate the PMS. Most importantly, they cause 'die' to be called in a subshell which is not only explicitly noted as unreliable by the PMS but also currently broken in pkgcore. Furthermore, PMS does not give any guarantees or allowance for helper functions to be used in parallel. In case of econf, this could cause real issues (and already did!) since the function modifies files in working directory. Historically, the multijob_* logic was also used to parallelize eautoreconf. However, that logic was removed in Nov 2014 as breaking stuff and never reintroduced. It was also used by multiprocessing & co to run sub-phases in parallel but it was removed as unreliable and too hard to support. All of the above considered, I believe we should phase out the multijob_* logic as an unsuccessful experiment. It's not used by any developer except for vapier, it's hard to use and it's a sure way to break stuff, as it has already proven multiple times.
I should probably point out that the new versions of media-tv/kodi no longer use this logic, so it is actually used by 4 packages.
Same goes for net-misc/suite3270 and sys-apps/sandbox. So it's actually actively used by two packages: dev-lang/jimtcl which hasn't been touched since 2015 and sys-libs/ncurses.
CC-ing all people whose packages are affected by the linked PR (i.e. packages using multijob* or affected by the related cleanup).
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=2df3277cb07eb3cddca3d3280a256394fb069e93 commit 2df3277cb07eb3cddca3d3280a256394fb069e93 Author: Michał Górny <mgorny@gentoo.org> AuthorDate: 2017-12-31 10:24:21 +0000 Commit: Michał Górny <mgorny@gentoo.org> CommitDate: 2017-12-31 21:48:45 +0000 sys-boot/grub: Stop using awful multijob_* logic The multijob logic adds a lot of complex, unmaintained and fragile bash code with little gain. Moreover, makes build logs unreadable by interspersing output from multiple configure runs. Bug: https://bugs.gentoo.org/613322 sys-boot/grub/grub-0.97-r16.ebuild | 10 +--------- sys-boot/grub/grub-0.97-r18.ebuild | 10 +--------- 2 files changed, 2 insertions(+), 18 deletions(-) https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=00ed64aebc52061bedaaf0511b041f388315beb9 commit 00ed64aebc52061bedaaf0511b041f388315beb9 Author: Michał Górny <mgorny@gentoo.org> AuthorDate: 2017-12-31 09:48:51 +0000 Commit: Michał Górny <mgorny@gentoo.org> CommitDate: 2017-12-31 21:48:06 +0000 sys-libs/ncurses: Stop using awful multijob_* logic The multijob logic adds a lot of complex, unmaintained and fragile bash code with little gain. Moreover, makes build logs unreadable by interspersing output from multiple configure runs. Bug: https://bugs.gentoo.org/613322 sys-libs/ncurses/ncurses-5.9-r101.ebuild | 11 ++++------- sys-libs/ncurses/ncurses-6.0-r1.ebuild | 9 +++------ 2 files changed, 7 insertions(+), 13 deletions(-)}
The bug has been closed via the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=0e50583c165feedf24152dae1973af88e0005191 commit 0e50583c165feedf24152dae1973af88e0005191 Author: Michał Górny <mgorny@gentoo.org> AuthorDate: 2017-12-31 10:29:42 +0000 Commit: Michał Górny <mgorny@gentoo.org> CommitDate: 2018-01-04 21:56:14 +0000 multiprocessing.eclass: Remove multijob_* functions Remove the multijob functions that were used to run bash code in parallel. The code was very complex, fragile and unmaintained. It has been used scarcely, and pretty much by a single developer. It gave very little gain, usually at the cost of losing readability and violating PMS. Closes: https://bugs.gentoo.org/613322 eclass/multiprocessing.eclass | 178 +++------------------------------------- eclass/tests/multiprocessing.sh | 42 ---------- 2 files changed, 10 insertions(+), 210 deletions(-) Additionally, it has been referenced in the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=398d7df6783c137e1021410de38a4379ca736c55 commit 398d7df6783c137e1021410de38a4379ca736c55 Author: Michał Górny <mgorny@gentoo.org> AuthorDate: 2017-12-31 10:04:26 +0000 Commit: Michał Górny <mgorny@gentoo.org> CommitDate: 2018-01-04 21:55:55 +0000 dev-lang/jimtcl: Stop using awful multijob_* logic The multijob logic adds a lot of complex, unmaintained and fragile bash code with little gain. Moreover, makes build logs unreadable by interspersing output from multiple configure runs. Bug: https://bugs.gentoo.org/613322 dev-lang/jimtcl/jimtcl-0.76.ebuild | 8 +++----- dev-lang/jimtcl/jimtcl-9999.ebuild | 8 +++----- 2 files changed, 6 insertions(+), 10 deletions(-)}
