Bugzilla Bug 56408

Currently, ebuild.sh is designed such that it sources the ebuild (and requisite
eclass's) for every stage- this is wasteful, and prone to lots of
situations/bugs that shouldn't be allowed.

Specifically, during removal-
A) portage's functions/returns need to be what the ebuild expected when it was
merged- any older ebuild's merged that expected use_enable to return 1 or 0
won't function correctly under .51 where 
use_enable returns 0 always (same for use_with).

B) the eclass could've changed *radically* since the ebuild was first emerged,
and there is no gurantee that the ebuild is compatible with the newer version
of the eclass.  Yes, yelling at people to 
maintain the eclass such that they are fully backwards compatible is probably
the first reaction, but this isn't a viable solution, let alone realistic.

General issues
A) during an actual emerge, multiple files on the fs are accessed at each
stage- this introduces the possibility of a typo in /etc/profile.env or
/etc/init.d/functions or /etc/rc.d/config/functions to cause a long running
emerge to bail out when it hits the next phase.  This behaviour isn't desired,
nor is it really right/sane- once an emerge has started, it's not safe/sane/fun
to have the possibility of the env/functions available changing.

B) ebuild.sh's current method of storing ebuild defined vars drops *all*
attributes- see bug #52652 for a real world example where it causes borkage.

C) ebuild.sh stomps on vars passed into it (think USE="blah" ebuild dar) via
sourcing.  See bug #51095 for background, bug #51552 for the actual request and
bug #51370 for a package this is causing user issues for.

D) All QA interceptors are currently disabled due to a simple typo in
ebuild.sh.  This is easily fixed, but there is no point in having these
functions defined (read: *used*) when ebuild.sh is not processing an ebuild in
a DEPEND phase.  Added, no point in defining a half dozen functions (and having
them in the env permenantly) that are exactly the same minus the function name;
this should be automated and generated via an eval; the QA interceptors should
also be able to be yanked from the env.

E) Do to the current layout of ebuild.sh, it's entirely possible for an
ebuild/eclass to overwrite internal ebuild.sh functions.  The ebuild.sh
internal functions *should* not be overwrote, and there is no point in letting
them be overwrote- functions can be flagged read only too.  Note this exempts a
set of functions the ebuild expects, src_unpack, src_compile, etc.  In general,
ebuild.sh expects certain functions to be sane (dyn_clean, dyn_setup, hasq,
etc) that the ebuild should never be touching.  So we should lock them down. 
This also would apply to eclasses.
To head off the "declaring blah readonly isn't going to make things any more
secure", marking internals functions as readonly isn't intended to make things
more secure.  It's intended to lock down the ebuild environment so that
debugging these beasts is easier, so that we can state 'well it has to be
screwing up in this function- everything else is locked down'.

F) Env vars can be changed between individual ebuild phases when using the
ebuild python script for debugging- this makes debugging potentially trickier. 
Been using BLAH="x" ebuild cpv.ebuild phase for testing, and you do notice you
forget the BLAH="x" definition for a phase- you've just changed the ebuild's
environment.  The environment defined/available in setup should be taken as the
initial snapshot, and propogated on through the phases.

G) the setup phase should not be wiping ${T} (this is specific to portage.py)-
that's what clean is for.  Additionally, setup should be ran only once when
stepping through phase's via the ebuild python.

Yeah... I think that's most of it.

Everything listed above I've addressed in my the ebuild.sh
rewrite/gutting/surgery; exempting 4 lines of modification to portage.py, it's
all bash env related.  It's also backwards compatible- since my version of
ebuild.sh locks down it's internals, there is no way for older saved envs from
installed ebuilds to hose up the current env.

Aside from that, the rewrote ebuild*.sh ought to be up to date w/ .51_pre12; it
was based off of pre10, and I've merged changes in by hand (only changes being
bug 53368 ebuild.sh patch inclusion).

PORTDIR_LOG has been intentionally backed out of ebuild.sh- this should be
moved up into spawn, abuse the piping ability.  Recursing ebuild.sh is kinda
nasty as a method to start up the logging :)

Created an attachment (id=34978) [details]
portage.py.patch

Patch to stop doebuild from wiping ${T}; this is clean's job.

Created an attachment (id=34979) [details]
ebuild.sh v1

ebuild.sh .  No point in submitting a diff, I've moved things around *way* too
much for the diff to be sane (although ignoring white space might make it
saner).

ebuild.sh is purely the arg processing, and env handling code.	All
non-essential functions have been moved out for the sake of keeping things neat
and clean.
Debug code is disabled, but currently left in place.

Created an attachment (id=34980) [details]
ebuild-default-functions.sh

ebuild.sh's internal functions; these are all marked readonly, and not saved in
the ebuild's env.
Think of it as ebuild.sh's internal/req'd functions.

Created an attachment (id=34981) [details]
ebuild-functions.sh

functions saved in the packages saved env.  use_with/use_enable, etc. 
Functions that may change, and *will* directly affect that ebuild down the
line.
These functions are able to change w/out issue- the functions defined in
-default-functions should maintain their return behaviours.

Basically think of this as ebuild.sh's api for ebuilds (horrid description, but
it'll do).

To give a general idea of how much abuse this code has been put through, keep
in mind this is a split up ebuild.sh w/ a lot of env loving applied- basically
I didn't change code unless I had to.  So large portions of the code have
already been tested prior to me shifting them around.

Additionally, I brought my system up from a stage1 to full
xorg-x11/flux/mozilla/etc using these patches.  The modifications have behaved
rather well.

/me shuts up after that flood of patches/files/comments :)

Created an attachment (id=35014) [details]
portage.py.patch v2

Uploaded an old patch that left out the fix for setup cleansing.
Now $T is only cleansed during the clean ebuild phase.

Created an attachment (id=35262) [details]
ebuild.sh v2

One line adjustment changing
source /etc/init.d/functions.sh
to
source /etc/init.d/functions.sh ${EBUILD_PHASE}

Otherwise, /etc/init.d/functions will make a portageq call, *horrendously*
slowing things down.

A suggestion, I think we should make sure that it is still possible to change
an ebuild and then just go on with the stage we were. This is very helpfull for
development (could be feature protected though), saving precious time unpacking
things for the umpteenth time, and preserving the compile history.

Similarly, ebuild might be coded such that in some cases changes in command
line useflags are incorporated.

I see no point in allowing use flags to change midway through the build
process, since it allows for the possibility of earlier phases assuming xyz to
be true, and later stages assuming zyx to be true.

The changes induced in cvs to protect the env are intended to ensure the build
process is exact in it's usage of the env, same for removal of packages.
Basically, these changes ensure things are correct, at the expense of a bit of
flexibility for the developer who is attempting to walk through each phase
continually adjusting the ebuild.

Personally, what I do is this- if I'm walking through the phases of an ebuild
w/ this code, and want to make a change, I just modify the saved environment. 
That is what's used, and is easy to tweak.  Test the changes w/in the saved
env, then add them into the ebuild.

Worth noting for those using portage-cvs or these patches (or the ebd
patchset), the detection of completed phases is no longer a collection of "does
blah exist"- /var/tmp/portage/${P}/.completed_stages holds the list of phases
that have finished.

As to why I *really* dislike this request, you're basically after making the
env non-linear via an opt.  building/merging of the package for the user -will-
be linear in env handling, as such, the dev should be testing the ebuild under
that setup.  They can always modify the saved env if they're walking the phases
and doing debugging...

*** Bug 42941 has been marked as a duplicate of this bug. ***

This is in portage cvs head (has been for about ~3 months).

Fixed on or before 2.0.51.22-r1 

Looking through the batch of bugs, I'm not sure that some of these are 
actually fixed in stable. Others, the requirements have possibly changed after 
the initial fix was committed. 
 
If you think this bug has been closed incorrectly, please reopen or ask that 
it be reopened. 

This bug is still an issue, although most of the dependencies have been worked
around in a fashion.
So... leave 'er closed, it's fixed in what's being worked on for next version.

Not fixed... it's not in stable.

*** Bug 110098 has been marked as a duplicate of this bug. ***