Bug 698400 – Portage 2.3.49: @system set scheduling constraints leak from BROOT to SYSROOT

• Product: Portage Development
• Status: CONFIRMED ---

[Description Chris McDonald 2019-10-23 16:33:38 UTC]

In a cross-compilation setup starting with a populated BROOT and a bare SYSROOT, adding packages to the @system set in the profile used by the BROOT causes parallelism for builds targeting the SYSROOT to drop to effectively 1 job at a time. The SYSROOT has an empty @system set.

Reproducible: Always

Actual Results:  
Adding packages to the @system set of the BROOT profile causes the build for a distinct SYSROOT to massively reduce parallelism and schedule packages sequentially as if they were part of the @system set for the target.

Expected Results:  
Packages in the @system set in the BROOT should not impose ordering constraints on packages to be installed to a separate SYSROOT. Since these packages are (potentially) being cross-compiled, they are not executed as part of the build process and should not be subjected to the scheduling constraints imposed by @system.

[Comment 1 Zac Medico 2019-11-26 20:28:14 UTC]

This constraint is intended to protect against issues similar to bug 256616.

[Comment 2 Zac Medico 2019-12-01 22:58:35 UTC]

We can add an option to make it less paranoid, and then it naturally will not account for cases like bug 256616.

The option suggested in bug 681312 would work, but that might not be paranoid enough, depending on your needs.

[Comment 3 Zac Medico 2019-12-08 20:33:02 UTC]

The --implicit-system-deps <y|n> option is now implemented (bug 681312):

https://gitweb.gentoo.org/proj/portage.git/commit/?id=ba871d9232b1dc1e0a8090f7664e05e23818b1b4


(In reply to Chris McDonald from comment #0)
> Actual Results:  
> Adding packages to the @system set of the BROOT profile causes the build for
> a distinct SYSROOT to massively reduce parallelism and schedule packages
> sequentially as if they were part of the @system set for the target.

This minimizes the impact of fragile states like the "/bin/sh: Command not found" reported in bug 256616, and states like that would affect SYSROOT builds just as much as BROOT builds.

> Expected Results:  
> Packages in the @system set in the BROOT should not impose ordering
> constraints on packages to be installed to a separate SYSROOT. Since these
> packages are (potentially) being cross-compiled, they are not executed as
> part of the build process and should not be subjected to the scheduling
> constraints imposed by @system.

If we assume that fragile states like the "/bin/sh: Command not found" reported in bug 256616 are practically negligible, the benefits of applying special @system scheduling constraints to BROOT becomes less obvious. Ideally, explicit dependencies would make it unnecessary to apply special @system scheduling constraints, and --implicit-system-deps=n would produce the desired behavior.

That said, it's obvious that SYSROOT @system packages should not affect scheduling constraints for BROOT. The fact that they currently do is an obvious bug that's worth fixing. While we're at it, we may as well add an option to prevent BROOT @system scheduling constraints from affecting SYSROOT (the benefit relative to --implicit-system-deps=n could be negligible but who knows?).

[Comment 4 Zac Medico 2019-12-08 21:00:36 UTC]

Note that the trigger for bug 259954, sys-libs/pam having PDEPEND="sys-auth/pambase", is another example (similar to bug 256616) of how it might be useful to have @system set scheduling constraints leak from BROOT to SYSROOT (but not vice versa).