Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 4980 - request for --almostemptytree=package type support
Summary: request for --almostemptytree=package type support
Status: RESOLVED DUPLICATE of bug 16342
Alias: None
Product: Portage Development
Classification: Unclassified
Component: Core (show other bugs)
Hardware: x86 Linux
: High enhancement (vote)
Assignee: Portage team
Depends on:
Reported: 2002-07-14 05:48 UTC by Olav Kolbu
Modified: 2011-10-30 22:37 UTC (History)
2 users (show)

See Also:
Package list:
Runtime testing required: ---


Note You need to log in before you can comment on or make changes to this bug.
Description Olav Kolbu 2002-07-14 05:48:57 UTC
I just did an 'emerge --emptytree world' on one of my slower machines, it's
still going after two days. Some of the reason for this was that it ran out of
space in /var/tmp when it hit X. So I changed the PORTAGE_TMPDIR to point to a
partition with more space. But at this point, I can't continue, there is no
support for doing partial emptytrees if you see what I mean. I had to re-emerge
all the packages that were emerged before X, _again_. I could have said "create
binary packages as you go", but I didn't. This takes space, and they wouldn't be
needed if it hadn't crashed.

So the suggestion is for an option that allows you to define just how "deep" you
want the emerge to start. The re-emerge is a linear thing, so this should be a
very easy option to implement for this particular scenario.

Such a feature would be very handy, not just for this scenario, but for the
other bug report I had the other day regarding emerge not re-emerging
dependencies not explicitly mentioned in the world-file. If say there is a new
libNNNNN out, that fixes a security hole, and I have lots of packages depending
on this lib. I want to be able to say " Now, I know there is a new libNNNNN out,
and I want to re-emerge this and every package that depends on this as some of
them may have linked it in statically and I don't know which ones". 

Implementation of this is a tad trickier, since the linarity of the dependencies
 break down. I will get lots of false positives where irrelevant packages have
been installed after my libNNNNN,and they will be re-emerged even tho they do
not depend on libNNNNN. I can live with this, as the only real choice today is a
"emerge --emptytree world", and this takes forever and his granmother. I'm
presuming here that the order packages appear in the world-file is strictly
adhered to so that it can be used to keep track of the order packages were
originally installed in. Otherwise if the system is too smart and tries to do
this by following dependencies only, we might see false negatives here. Where
say a changed USE-setting makes a package that used to be dependent on libNNNNN
no longer so and it will stay "vulnerable".

Please send me a comment if you decide to scrap this idea, as I very much think
this is needed. :-)
Whether the option should mean "re-emerge everything above this package,
_including said package_". Naming I'll leave up to you. :-)
Comment 1 Tom von Schwerdtner 2002-07-27 10:00:33 UTC
I was just thinking of something along the lines of: 
emerge --emptytree=3 xfree 
Where it would emerge xfree, and 3 levels of dependancies. 
So, for example, "emerge --emptytree=1 xfree" would merge X and everythign that 
X depends on.  --emptytree=2 would merge X, everything it depends on, and 
everything its depends depend on (and stop there). 
It would also be nice to be able to exclude packages: 
emerge --emptytree=3 xfree --exclude freetype2 
or maybe even: 
emerge --emptytree=3 xfree --exclude xfree 
which would build the xfree depends down 3 levels, but not xfree itself. 
Comment 2 Olav Kolbu 2002-07-29 10:54:43 UTC
A couple of problems with this "--emptytree=3 xfree" proposal. 

First off it's not easy to view the non-flattened dependency graph, so people
are very much looking at the dependency graph as a linear list of dependencies.
 "Easy" here meaning there isn't a single command to do it, you'd have to run
emerge --pretend --emptytree ... several times with different package names to
get the whole picture. At least afaik. Hm, as an aside here, how about a mode
where emerge output indents the list of packages depending on dependency. Should
be a 1-2 line hack, i.e. just print # of spaces equal to # of levels of
recursion you're in at that time (depending slightly on traversal algorithm).
Off by default of course, I think Joe User is better off seeing it as a flat
list to keep the complexity down to a minimum.

The other problem, again depends on what order emerge does dependencies.
Presumably it does depth first, covering all leaf nodes before the central node. 

Let's say xfree has four dependency "legs", and my update failed somewhere way
down in the third dependency leg. I.e. leg 1 and 2 have been completely
re-emerged. To continue this emerge in an efficient manner given your suggested
options, I'd have to find the top package of each of the first two legs, add
them to the --exclude list (hoping that this also excludes everything below) and
then set my --emptytree=# to how far down leg 3 I was at the time. This means a
lot of fiddling about to find the right packages and levels. So for my scenario,
I can't see much use for a fixed cutoff-number, seeing that the "legs" are bound
to be of varying lengths. And especially since they're largely independent of
eachother, the number of nodes I need to go down in one particular leg generally
doesn't mean that I want to go down the same number of nodes in any of the other
legs. Rather the opposite I'd say. And for someone who is used to viewing the
graph as a list, this quickly becomes confusing and error prone.

By all means, add the feature. But for my particular problem I think the best
thing to do is to treat the dependency graph as one flat, linear list of
packages. And in this scenario provide people with the ability to specify a
"From"-package, the "To"-package being given by default as the actual package
you want to (re)emerge. This is basically what --emptytree does, but it's
unfortunately hardwired.

Note that there have to be a certain number of checks to make sure the user
doesn't mess up his dependencies. I.e. I should not be able to get going if I've
set my cutoff "above" some package whose new version is needed by a package
above my cutoff. I suppose this is just a matter doing the checks after the list
has been pruned, not before.

Comment 3 Alain Penders (RETIRED) gentoo-dev 2003-02-02 11:44:55 UTC
I think the better solution for this problem is to remember the emerge that was in progress.  If portage saves all information about the emerge it's doing, then it can reload this information, and continue the emerge where it left off after a failure.

Seems like a cleaner solution to me than to try to remember which parts of the tree to remove to continue an install at a specific point.

I also don't like the --emptytree=<number> approach, as it'd turn into a trial and error approach using --pretend to figure out what exactly it will do.
Comment 4 Ryan Kelly 2003-07-30 21:18:25 UTC
I've just run into a similar problem when my `emerge -e world` was ruined after two days by coreutils failing.  Luckily I recorded the output of `emerge -ep world` and with a bit of shell script trickery I was able to continue from this list.

In my humble opinion, the neatest solution would be for emerge to record its actions (tiemstamped, or course) as it goes, perhaps somewhere under /var.  It would list all the packages it's about the emerge, and keep track of the ones it has done.
If it completes, or is interupted, it gets rid of the record.  If it bails because of an error, a record of precisely where it was up to is retained.  I could then come back later and do `emerge resume` to pick up where I left off on any failed emerges.

A little off topic now, but it might even be handy to be able to `emerge pause` and hang all currently running emerge instances (for example, if I want a little responsiveness for a while) until I `emerge resume` to get it going again.

The advantage of keeping it in the filesystem is that we can use other tools to analyse it if something goes REALLY wrong.
Comment 5 Marius Mauch (RETIRED) gentoo-dev 2003-10-11 01:18:51 UTC
emerge --resume should take care of this.
Comment 6 Olav Kolbu 2003-10-13 03:54:57 UTC
> emerge --resume should take care of this.

No it doesn't. The original question here was for a more generic feature
of allowing emerge to start off at a given point in the emerge graph, and
have it build everything above that. Something you would do when say a critical
library was updated and you had no idea which programs used it. As a bonus,
this could also be used if your "emerge a-lot-of-stuff" failed half way through.
The --resume would presumably help with the last example there, but who really
knows. It's way too easy to trigger a crash using resume:

bash-2.05b# emerge --emptytree --update world
Calculating world dependencies ...done!

!!! Error: the x11-libs/xft package conflicts with another package.
!!!        both can't be installed on the same system together.
!!!        Please use 'emerge --pretend' to determine blockers.

bash-2.05b# emerge --resume world
Traceback (most recent call last):
  File "/usr/bin/emerge", line 2143, in ?
KeyError: mergelist

So. Either close the request saying this can not be done/we can't be bothered,
or keep it open until someone can actually add such a feature. Thank you.
Comment 7 Brian Harring (RETIRED) gentoo-dev 2004-04-08 19:54:42 UTC
Technically an enhancement, so marking it as such...
Comment 8 Marius Mauch (RETIRED) gentoo-dev 2004-08-03 12:29:31 UTC

*** This bug has been marked as a duplicate of 16342 ***