Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 387011 - sys-apps/portage: document that emerge --prune removes atoms from the world file
Summary: sys-apps/portage: document that emerge --prune removes atoms from the world file
Status: RESOLVED FIXED
Alias: None
Product: Portage Development
Classification: Unclassified
Component: Core (show other bugs)
Hardware: All Linux
: Normal normal (vote)
Assignee: Portage team
URL:
Whiteboard:
Keywords: InVCS
Depends on:
Blocks: 381649
  Show dependency tree
 
Reported: 2011-10-13 06:59 UTC by Sebastian Luther (few)
Modified: 2011-10-17 06:31 UTC (History)
1 user (show)

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Sebastian Luther (few) 2011-10-13 06:59:33 UTC
This option is a completely broken idea with no valid use cases. People should be using --depclean. 

Any objections against removing it?
Comment 1 Zac Medico gentoo-dev 2011-10-13 14:53:01 UTC
It allows to check for and remove redundant slots. What's wrong with that? We don't have any other action that does this.
Comment 2 Zac Medico gentoo-dev 2011-10-13 14:54:54 UTC
To clarify, --prune will find the redundant slots even if they are pulled in by your world file, and --depclean does not have a way to do that.
Comment 3 Sebastian Luther (few) 2011-10-13 15:00:39 UTC
Why would they be redundant if a world update pulls them in?
Comment 4 Zac Medico gentoo-dev 2011-10-13 15:02:26 UTC
You might decide that you want to remove some slot atoms from your world file. Without --prune, you would have to manually search your world file for slot atoms in order to do this.
Comment 5 Sebastian Luther (few) 2011-10-13 15:46:17 UTC
Ok one thing about this where I was wrong is I thought that it would always only keep the highest slot, even if things depend on lower slots. This is works just as it should. (What is the warning in the man page about then btw?)

Remains the question what it really does. Playing with it shows that it would remove packages wich are pulled in by world, doing this even if this was the last entry in world for this package.

Another thing I noticed that it doesn't remove the highest slot, even if it is redundant (all deps are slot deps on lower slots).

So what is this thing really supposed to do?
Comment 6 Zac Medico gentoo-dev 2011-10-13 16:10:37 UTC
(In reply to comment #5)
> Ok one thing about this where I was wrong is I thought that it would always
> only keep the highest slot, even if things depend on lower slots. This is works
> just as it should. (What is the warning in the man page about then btw?)

I guess that warning may be left over from the days when --prune didn't account for dependencies. It used to only behave like it does now with --nodeps.

> Remains the question what it really does. Playing with it shows that it would
> remove packages wich are pulled in by world, doing this even if this was the
> last entry in world for this package.
> 
> Another thing I noticed that it doesn't remove the highest slot, even if it is
> redundant (all deps are slot deps on lower slots).
> 
> So what is this thing really supposed to do?

The main idea is to locate lower slots that can be removed (even if they happen to have matching slot atoms listed in the world file).
Comment 7 Michael Orlitzky gentoo-dev 2011-10-13 17:18:37 UTC
It's worth keeping if only for the removal of old kernels. It would suck having to type out,

emerge -vC hardened-sources-2.6.32-r1
emerge -vC hardened-sources-2.6.32-r4
emerge -vC hardened-sources-2.6.32-r9
emerge -vC hardened-sources-2.6.34-r2
emerge -vC hardened-sources-2.6.34-r10
emerge -vC hardened-sources-2.6.36-r5
emerge -vC hardened-sources-2.6.37
emerge -vC hardened-sources-2.6.37-r6
...

when I'm at 3.0.4-r1.
Comment 8 Sebastian Luther (few) 2011-10-13 17:48:34 UTC
(In reply to comment #6)
> (In reply to comment #5)
> > Remains the question what it really does. Playing with it shows that it would
> > remove packages wich are pulled in by world, doing this even if this was the
> > last entry in world for this package.
> > 
> > Another thing I noticed that it doesn't remove the highest slot, even if it is
> > redundant (all deps are slot deps on lower slots).
> > 
> > So what is this thing really supposed to do?
> 
> The main idea is to locate lower slots that can be removed (even if they happen
> to have matching slot atoms listed in the world file).

The question is if this is really a good idea considering that it may remove the last atom from world for this package.

I started this discussion because someone was about to suggest --prune for the removal of old kernel sources in the wiki. Considering that it may remove the kernel sources completely from world, using --prune for cleaning up old slots is imo problematic. 

Imo the whole use case you described in comment 4 is questionable. Why would you want emerge to randomly remove slot atoms from your world file? If you don't want them there, don't put them there in the first place. This seems like a bug in the user's work flow, nothing that warrens an option.
Thinking this further it might make it even worse. Because if the user has slot atoms in world which shouldn't be there, he probably has atoms for packages in there which aren't slots atoms, but still shouldn't be in world.
The only way to fix is to manually inspect the world file.


(In reply to comment #7)
> It's worth keeping if only for the removal of old kernels. It would suck having
> to type out,
> 
> emerge -vC hardened-sources-2.6.32-r1
> [...]

a) Why did you add them to the world file?
Assuming these commands are a replacement for what --prune does:
b) emerge -vC <=hardened-sources-2.6.37-r6 would do what all your commands together do

Keeping it only for kernel sources doesn't work because it works for all slotted packages and what works will be used by someone no matter how much sense this makes.
Comment 9 Michael Orlitzky gentoo-dev 2011-10-13 18:05:06 UTC
> 
> (In reply to comment #7)
> > It's worth keeping if only for the removal of old kernels. It would suck having
> > to type out,
> > 
> > emerge -vC hardened-sources-2.6.32-r1
> > [...]
> 
> a) Why did you add them to the world file?
> Assuming these commands are a replacement for what --prune does:

They aren't in the world file, sys-kernel/hardened-sources is. Whenever a new version is available, it gets emerged in a new slot.


> b) emerge -vC <=hardened-sources-2.6.37-r6 would do what all your commands
> together do

Sure, if I know what versions are installed and which ones are old. A lot of the time I'll emerge kernels that never get compiled and they just sit there.

  emerge -pv --prune

tells me that, and then gets rid of them for me.


> Keeping it only for kernel sources doesn't work because it works for all
> slotted packages and what works will be used by someone no matter how much
> sense this makes.

It also catches old versions of automake, documentation utilities, database servers, PHP, etc. I don't understand the potential problems that you bring up so I won't comment on them; but, there are legitimate uses.
Comment 10 Sebastian Luther (few) 2011-10-13 18:13:02 UTC
For all the cases you mentioned --depclean will do the same as --prune. As stated in comment 0, use --depclean instead.
Comment 11 Michael Orlitzky gentoo-dev 2011-10-13 18:19:09 UTC
(In reply to comment #10)
> For all the cases you mentioned --depclean will do the same as --prune. As
> stated in comment 0, use --depclean instead.

Not true. Right now, on one of our mail servers:

# emerge -p --prune

Calculating dependencies... done!
>>> Calculating removal order...

>>> These are the packages that would be unmerged:

 sys-kernel/hardened-sources
    selected: 2.6.38-r6 
   protected: none 
     omitted: 3.0.4-r1 

 sys-kernel/hardened-sources
    selected: 2.6.37-r7 
   protected: none 
     omitted: 3.0.4-r1 

 sys-kernel/hardened-sources
    selected: 2.6.36-r9 
   protected: none 
     omitted: 3.0.4-r1 

 sys-devel/gcc
    selected: 4.4.5 
   protected: none 
     omitted: 4.5.3-r1 

 sys-kernel/hardened-sources
    selected: 2.6.39-r8 
   protected: none 
     omitted: 3.0.4-r1 

 dev-db/postgresql-server
    selected: 8.4.9 
   protected: none 
     omitted: 9.0.5 

 dev-db/postgresql-base
    selected: 8.4.9 
   protected: none 
     omitted: 9.0.5


# emerge -p --depclean

 * Depclean may break link level dependencies. Thus, it is
 * recommended to use a tool such as `revdep-rebuild` (from
 * app-portage/gentoolkit) in order to detect such breakage.
 * 
 * Always study the list of packages to be cleaned for any obvious
 * mistakes. Packages that are part of the world set will always
 * be kept.  They can be manually added to this set with
 * `emerge --noreplace <atom>`.  Packages that are listed in
 * package.provided (see portage(5)) will be removed by
 * depclean, even if they are part of the world set.
 * 
 * As a safety measure, depclean will not remove any packages
 * unless *all* required dependencies have been resolved.  As a
 * consequence, it is often necessary to run `emerge --update
 * --newuse --deep @world` prior to depclean.

Calculating dependencies... done!
>>> No packages selected for removal by depclean
>>> To see reverse dependencies, use --verbose
Packages installed:   234
Packages in world:    108
Packages in system:   46
Required packages:    234
Number to remove:     0
Comment 12 Sebastian Luther (few) 2011-10-13 18:22:23 UTC
Try --depclean --with-bdeps=n. They are probably build time only deps of some installed package, which aren't removed by depclean.
Comment 13 Michael Orlitzky gentoo-dev 2011-10-13 18:25:41 UTC
(In reply to comment #12)
> Try --depclean --with-bdeps=n. They are probably build time only deps of some
> installed package, which aren't removed by depclean.

# emerge -p --depclean --with-bdeps=n

 * Depclean may break link level dependencies. Thus, it is
 * recommended to use a tool such as `revdep-rebuild` (from
 * app-portage/gentoolkit) in order to detect such breakage.
 * 
 * Always study the list of packages to be cleaned for any obvious
 * mistakes. Packages that are part of the world set will always
 * be kept.  They can be manually added to this set with
 * `emerge --noreplace <atom>`.  Packages that are listed in
 * package.provided (see portage(5)) will be removed by
 * depclean, even if they are part of the world set.
 * 
 * As a safety measure, depclean will not remove any packages
 * unless *all* required dependencies have been resolved.  As a
 * consequence, it is often necessary to run `emerge --update
 * --newuse --deep @world` prior to depclean.

Calculating dependencies... done!
>>> Calculating removal order...

>>> These are the packages that would be unmerged:

 sys-devel/gettext
    selected: 0.18.1.1-r1 
   protected: none 
     omitted: none 

 sys-devel/automake
    selected: 1.11.1 
   protected: none 
     omitted: none 

 app-portage/portage-utils
    selected: 0.3.1 
   protected: none 
     omitted: none 

 dev-util/gtk-doc-am
    selected: 1.17 
   protected: none 
     omitted: none 

 sys-devel/automake
    selected: 1.10.3 
   protected: none 
     omitted: none 

 virtual/perl-Module-Build
    selected: 0.36.07 
   protected: none 
     omitted: none 

 virtual/yacc
    selected: 0 
   protected: none 
     omitted: none 

 perl-core/Module-Build
    selected: 0.36.07 
   protected: none 
     omitted: none 

 sys-devel/bison
    selected: 2.4.3 
   protected: none 
     omitted: none 

 sys-devel/automake-wrapper
    selected: 4 
   protected: none 
     omitted: none 

 sys-devel/autoconf
    selected: 2.68 
   protected: none 
     omitted: none 

 virtual/perl-ExtUtils-ParseXS
    selected: 2.22.05 
   protected: none 
     omitted: none 

 dev-perl/YAML-Tiny
    selected: 1.41 
   protected: none 
     omitted: none 

 virtual/perl-Archive-Tar
    selected: 1.54 
   protected: none 
     omitted: none 

 virtual/perl-Test-Harness
    selected: 3.17 
   protected: none 
     omitted: none 

 app-emacs/autoconf-mode
    selected: 2.68 
   protected: none 
     omitted: none 

 perl-core/ExtUtils-ParseXS
    selected: 2.22.05 
   protected: none 
     omitted: none 

 virtual/perl-ExtUtils-CBuilder
    selected: 0.27.03 
   protected: none 
     omitted: none 

 perl-core/ExtUtils-CBuilder
    selected: 0.27.03 
   protected: none 
     omitted: none 

All selected packages: sys-devel/automake-1.11.1 virtual/perl-ExtUtils-CBuilder-0.27.03 app-emacs/autoconf-mode-2.68 virtual/yacc-0 sys-devel/bison-2.4.3 sys-devel/gettext-0.18.1.1-r1 perl-core/Module-Build-0.36.07 sys-devel/automake-1.10.3 virtual/perl-Test-Harness-3.17 virtual/perl-Module-Build-0.36.07 virtual/perl-ExtUtils-ParseXS-2.22.05 app-portage/portage-utils-0.3.1 perl-core/ExtUtils-CBuilder-0.27.03 sys-devel/automake-wrapper-4 sys-devel/autoconf-2.68 perl-core/ExtUtils-ParseXS-2.22.05 virtual/perl-Archive-Tar-1.54 dev-perl/YAML-Tiny-1.41 dev-util/gtk-doc-am-1.17

>>> 'Selected' packages are slated for removal.
>>> 'Protected' and 'omitted' packages will not be removed.

Packages installed:   233
Packages in world:    107
Packages in system:   46
Required packages:    214
Number to remove:     19
Comment 14 Sebastian Luther (few) 2011-10-13 18:38:52 UTC
Well, then please try again with --verbose added and search for those package you expect to be cleaned and show us the reverse deps it prints.
Comment 15 Michael Orlitzky gentoo-dev 2011-10-13 18:45:22 UTC
(In reply to comment #14)
> Well, then please try again with --verbose added and search for those package
> you expect to be cleaned and show us the reverse deps it prints.

I don't have to; prune and depclean just don't work the way you think they do. For example, I have dev-db/postgresql-server in my world file. So, it won't be depcleaned. The older slot can however be pruned.

While --depclean is "safe", --prune is very much not. Again as an example, I haven't migrated my data from postgres-8.x to 9.x on that machine yet, so it would be a huge mistake to prune it.
Comment 16 Zac Medico gentoo-dev 2011-10-13 18:55:25 UTC
(In reply to comment #15)
> (In reply to comment #14)
> > Well, then please try again with --verbose added and search for those package
> > you expect to be cleaned and show us the reverse deps it prints.
> 
> I don't have to; prune and depclean just don't work the way you think they do.
> For example, I have dev-db/postgresql-server in my world file. So, it won't be
> depcleaned. The older slot can however be pruned.

Depclean is supposed to remove the older slot if it has no reverse dependencies. So, the --verbose output would help us understand why it's not removed.
Comment 17 Michael Orlitzky gentoo-dev 2011-10-13 19:06:57 UTC
(In reply to comment #16)
> 
> Depclean is supposed to remove the older slot if it has no reverse
> dependencies. So, the --verbose output would help us understand why it's not
> removed.

Here are the relevant ones. I already pruned gcc-4.4.5, sorry. I didn't realize this was going to be a big deal =)

I have a few more servers I'm updating over the next few days so I'm sure I'll have more examples.


  dev-db/postgresql-base-8.4.9 pulled in by:
    @selected
    dev-db/postgresql-server-8.4.9

  dev-db/postgresql-server-8.4.9 pulled in by:
    @selected

  sys-kernel/hardened-sources-2.6.36-r9 pulled in by:
    @selected

  sys-kernel/hardened-sources-2.6.37-r7 pulled in by:
    @selected

  sys-kernel/hardened-sources-2.6.38-r6 pulled in by:
    @selected

  sys-kernel/hardened-sources-2.6.39-r8 pulled in by:
    @selected

  sys-kernel/hardened-sources-3.0.4-r1 pulled in by:
    @selected
Comment 18 Zac Medico gentoo-dev 2011-10-13 19:17:38 UTC
(In reply to comment #17)
>   dev-db/postgresql-base-8.4.9 pulled in by:
>     @selected
>     dev-db/postgresql-server-8.4.9
> 
>   dev-db/postgresql-server-8.4.9 pulled in by:
>     @selected

This is likely due to  dev-db/postgresql-server:8.4 atom in your world file, which --prune would ignore.
Comment 19 Michael Orlitzky gentoo-dev 2011-10-13 19:52:52 UTC
(In reply to comment #18)
> (In reply to comment #17)
> >   dev-db/postgresql-base-8.4.9 pulled in by:
> >     @selected
> >     dev-db/postgresql-server-8.4.9
> > 
> >   dev-db/postgresql-server-8.4.9 pulled in by:
> >     @selected
> 
> This is likely due to  dev-db/postgresql-server:8.4 atom in your world file,
> which --prune would ignore.

Precisely why --prune is useful =)
Comment 20 Sebastian Luther (few) 2011-10-13 20:11:57 UTC
(In reply to comment #19)
> (In reply to comment #18)
[...]
> > This is likely due to  dev-db/postgresql-server:8.4 atom in your world file,
> > which --prune would ignore.
> 
> Precisely why --prune is useful =)

Exactly the opposite. This is exactly the case I mentioned in comment 8.
Comment 21 Michael Orlitzky gentoo-dev 2011-10-13 20:18:57 UTC
(In reply to comment #20)
> 
> Exactly the opposite. This is exactly the case I mentioned in comment 8.

And the alternative is to dig through the world file by hand.
Comment 22 Zac Medico gentoo-dev 2011-10-13 20:20:04 UTC
(In reply to comment #8)
> I started this discussion because someone was about to suggest --prune for the
> removal of old kernel sources in the wiki. Considering that it may remove the
> kernel sources completely from world, using --prune for cleaning up old slots
> is imo problematic. 

I don't think this is much of a problem, since the user can use --depclean to check if there's anything missing from the world file.

> Imo the whole use case you described in comment 4 is questionable. Why would
> you want emerge to randomly remove slot atoms from your world file?

It's not random, it's the old slots. People add older slots to the world file sometimes, and later on it's nice for them to be able to use --pretend --prune to see what older slots are eligible for removal.
Comment 23 Sebastian Luther (few) 2011-10-14 10:43:48 UTC
(In reply to comment #22)
> (In reply to comment #8)
> I don't think this is much of a problem, since the user can use --depclean to
> check if there's anything missing from the world file.

Again the exactly opposite. The user shouldn't have to watch which packages depclean wants to remove. It should never remove anything the user put in his world file, except he did explicitly ask for it to be removed from the world file. --prune doesn't qualify as "the user ask for it to be removed from world".

Anyway, if you want to keep your bad habit so be it. I'll send you a man page update.
Comment 24 Zac Medico gentoo-dev 2011-10-14 14:59:42 UTC
(In reply to comment #23)
> I'll send you a man page update.

Thanks, this is in git:

http://git.overlays.gentoo.org/gitweb/?p=proj/portage.git;a=commit;h=d60105f611515f32b53b1eb4afa65f3a0d8a9056
Comment 25 Zac Medico gentoo-dev 2011-10-17 06:31:23 UTC
This is fixed in 2.1.10.28 and 2.2.0_alpha68.