so far, pretty much every pkg_pretend use in the tree can be categorized into: * filesystem size checks * USE flag checks * kernel config checks in CrOS, none of these things matter. in general, when cross-compiling, or doing large image building (i.e. in a CI environment), these things aren't that useful. that means running all the pkg_pretend phases up front before actually merging things slows down the overall build with no gain. can we add a --skip-pkg-pretend or similar option to emerge ? we've just patched the portage code to return immediately in the pkg_pretend phase.
(In reply to SpanKY from comment #0) > so far, pretty much every pkg_pretend use in the tree can be categorized > into: > * filesystem size checks > * USE flag checks > * kernel config checks > > in CrOS, none of these things matter. in general, when cross-compiling, or > doing large image building (i.e. in a CI environment), these things aren't > that useful. that means running all the pkg_pretend phases up front before > actually merging things slows down the overall build with no gain. > > can we add a --skip-pkg-pretend or similar option to emerge ? we've just > patched the portage code to return immediately in the pkg_pretend phase. I was hoping bashrc would handle it, but at first glance it appears to be sourced before ebuilds (not after.) If we look at phase-functions.sh: __dyn_pretend() { if [[ -e $PORTAGE_BUILDDIR/.pretended ]] ; then __vecho ">>> It appears that '$PF' is already pretended; skipping." __vecho ">>> Remove '$PORTAGE_BUILDDIR/.pretended' to force pretend." return 0 fi __ebuild_phase pre_pkg_pretend __ebuild_phase pkg_pretend >> "$PORTAGE_BUILDDIR/.pretended" || \ die "Failed to create $PORTAGE_BUILDDIR/.pretended" __ebuild_phase post_pkg_pretend } You could try to use your bashrc to arrange for '$PORTAGE_BUILDDIR/.pretended' to exist, and then it would skip the phase, right? You'd be carrying a bashrc instead of a patch; not sure if one is better vs the other? -A
(In reply to Alec Warner from comment #1) unfortunately that wouldn't suffice to fix the pipeline bubble. portage fetches the binpkgs & runs the pkg_pretend steps before moving on to the next set of steps. hacking it at the bashrc step would require those fetches to finish. by nerfing the entire step in portage, we can go back to fetching & installing binpkgs in parallel.
(In reply to SpanKY from comment #2) > (In reply to Alec Warner from comment #1) > > unfortunately that wouldn't suffice to fix the pipeline bubble. portage > fetches the binpkgs & runs the pkg_pretend steps before moving on to the > next set of steps. hacking it at the bashrc step would require those > fetches to finish. > > by nerfing the entire step in portage, we can go back to fetching & > installing binpkgs in parallel. Ah I was confused you had said you had 'patched emerge to return in the pkg_pretend phase' so I was looking for a similar less-patchy method. Can you attach your patch so we can see what you are currently using? Does your patch also solve for the above case where it blocks fetches running your empty pkg_pretend? -A
(In reply to Alec Warner from comment #3) i don't give it a chance to run any code at all, so there are no pipeline issues https://chromium.googlesource.com/chromiumos/third_party/portage_tool/+/dd1033474678f08f981e53afb24221654c1f8ff3%5E%21/#F0 it's very much a hack that works for us in CrOS, but obviously doesn't make sense upstream as-is
