Currently, ebuild.sh is designed such that it sources the ebuild (and requisite eclass's) for every stage- this is wasteful, and prone to lots of situations/bugs that shouldn't be allowed. Specifically, during removal- A) portage's functions/returns need to be what the ebuild expected when it was merged- any older ebuild's merged that expected use_enable to return 1 or 0 won't function correctly under .51 where use_enable returns 0 always (same for use_with). B) the eclass could've changed *radically* since the ebuild was first emerged, and there is no gurantee that the ebuild is compatible with the newer version of the eclass. Yes, yelling at people to maintain the eclass such that they are fully backwards compatible is probably the first reaction, but this isn't a viable solution, let alone realistic. General issues A) during an actual emerge, multiple files on the fs are accessed at each stage- this introduces the possibility of a typo in /etc/profile.env or /etc/init.d/functions or /etc/rc.d/config/functions to cause a long running emerge to bail out when it hits the next phase. This behaviour isn't desired, nor is it really right/sane- once an emerge has started, it's not safe/sane/fun to have the possibility of the env/functions available changing. B) ebuild.sh's current method of storing ebuild defined vars drops *all* attributes- see bug #52652 for a real world example where it causes borkage. C) ebuild.sh stomps on vars passed into it (think USE="blah" ebuild dar) via sourcing. See bug #51095 for background, bug #51552 for the actual request and bug #51370 for a package this is causing user issues for. D) All QA interceptors are currently disabled due to a simple typo in ebuild.sh. This is easily fixed, but there is no point in having these functions defined (read: *used*) when ebuild.sh is not processing an ebuild in a DEPEND phase. Added, no point in defining a half dozen functions (and having them in the env permenantly) that are exactly the same minus the function name; this should be automated and generated via an eval; the QA interceptors should also be able to be yanked from the env. E) Do to the current layout of ebuild.sh, it's entirely possible for an ebuild/eclass to overwrite internal ebuild.sh functions. The ebuild.sh internal functions *should* not be overwrote, and there is no point in letting them be overwrote- functions can be flagged read only too. Note this exempts a set of functions the ebuild expects, src_unpack, src_compile, etc. In general, ebuild.sh expects certain functions to be sane (dyn_clean, dyn_setup, hasq, etc) that the ebuild should never be touching. So we should lock them down. This also would apply to eclasses. To head off the "declaring blah readonly isn't going to make things any more secure", marking internals functions as readonly isn't intended to make things more secure. It's intended to lock down the ebuild environment so that debugging these beasts is easier, so that we can state 'well it has to be screwing up in this function- everything else is locked down'. F) Env vars can be changed between individual ebuild phases when using the ebuild python script for debugging- this makes debugging potentially trickier. Been using BLAH="x" ebuild cpv.ebuild phase for testing, and you do notice you forget the BLAH="x" definition for a phase- you've just changed the ebuild's environment. The environment defined/available in setup should be taken as the initial snapshot, and propogated on through the phases. G) the setup phase should not be wiping ${T} (this is specific to portage.py)- that's what clean is for. Additionally, setup should be ran only once when stepping through phase's via the ebuild python. Yeah... I think that's most of it. Everything listed above I've addressed in my the ebuild.sh rewrite/gutting/surgery; exempting 4 lines of modification to portage.py, it's all bash env related. It's also backwards compatible- since my version of ebuild.sh locks down it's internals, there is no way for older saved envs from installed ebuilds to hose up the current env. Aside from that, the rewrote ebuild*.sh ought to be up to date w/ .51_pre12; it was based off of pre10, and I've merged changes in by hand (only changes being bug 53368 ebuild.sh patch inclusion). PORTDIR_LOG has been intentionally backed out of ebuild.sh- this should be moved up into spawn, abuse the piping ability. Recursing ebuild.sh is kinda nasty as a method to start up the logging :)
Created attachment 34978 [details, diff] portage.py.patch Patch to stop doebuild from wiping ${T}; this is clean's job.
Created attachment 34979 [details] ebuild.sh v1 ebuild.sh . No point in submitting a diff, I've moved things around *way* too much for the diff to be sane (although ignoring white space might make it saner). ebuild.sh is purely the arg processing, and env handling code. All non-essential functions have been moved out for the sake of keeping things neat and clean. Debug code is disabled, but currently left in place.
Created attachment 34980 [details] ebuild-default-functions.sh ebuild.sh's internal functions; these are all marked readonly, and not saved in the ebuild's env. Think of it as ebuild.sh's internal/req'd functions.
Created attachment 34981 [details] ebuild-functions.sh functions saved in the packages saved env. use_with/use_enable, etc. Functions that may change, and *will* directly affect that ebuild down the line. These functions are able to change w/out issue- the functions defined in -default-functions should maintain their return behaviours. Basically think of this as ebuild.sh's api for ebuilds (horrid description, but it'll do).
To give a general idea of how much abuse this code has been put through, keep in mind this is a split up ebuild.sh w/ a lot of env loving applied- basically I didn't change code unless I had to. So large portions of the code have already been tested prior to me shifting them around. Additionally, I brought my system up from a stage1 to full xorg-x11/flux/mozilla/etc using these patches. The modifications have behaved rather well. /me shuts up after that flood of patches/files/comments :)
Created attachment 35014 [details, diff] portage.py.patch v2 Uploaded an old patch that left out the fix for setup cleansing. Now $T is only cleansed during the clean ebuild phase.
Created attachment 35262 [details] ebuild.sh v2 One line adjustment changing source /etc/init.d/functions.sh to source /etc/init.d/functions.sh ${EBUILD_PHASE} Otherwise, /etc/init.d/functions will make a portageq call, *horrendously* slowing things down.
A suggestion, I think we should make sure that it is still possible to change an ebuild and then just go on with the stage we were. This is very helpfull for development (could be feature protected though), saving precious time unpacking things for the umpteenth time, and preserving the compile history. Similarly, ebuild might be coded such that in some cases changes in command line useflags are incorporated.
I see no point in allowing use flags to change midway through the build process, since it allows for the possibility of earlier phases assuming xyz to be true, and later stages assuming zyx to be true. The changes induced in cvs to protect the env are intended to ensure the build process is exact in it's usage of the env, same for removal of packages. Basically, these changes ensure things are correct, at the expense of a bit of flexibility for the developer who is attempting to walk through each phase continually adjusting the ebuild. Personally, what I do is this- if I'm walking through the phases of an ebuild w/ this code, and want to make a change, I just modify the saved environment. That is what's used, and is easy to tweak. Test the changes w/in the saved env, then add them into the ebuild. Worth noting for those using portage-cvs or these patches (or the ebd patchset), the detection of completed phases is no longer a collection of "does blah exist"- /var/tmp/portage/${P}/.completed_stages holds the list of phases that have finished. As to why I *really* dislike this request, you're basically after making the env non-linear via an opt. building/merging of the package for the user -will- be linear in env handling, as such, the dev should be testing the ebuild under that setup. They can always modify the saved env if they're walking the phases and doing debugging...
*** Bug 42941 has been marked as a duplicate of this bug. ***
This is in portage cvs head (has been for about ~3 months).
Fixed on or before 2.0.51.22-r1
Looking through the batch of bugs, I'm not sure that some of these are actually fixed in stable. Others, the requirements have possibly changed after the initial fix was committed. If you think this bug has been closed incorrectly, please reopen or ask that it be reopened.
This bug is still an issue, although most of the dependencies have been worked around in a fashion. So... leave 'er closed, it's fixed in what's being worked on for next version.
Not fixed... it's not in stable.
*** Bug 110098 has been marked as a duplicate of this bug. ***
