Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 727576 - app-portage/gentoolkit eclean-pkg should have a flag to keep only one copy of each package
Summary: app-portage/gentoolkit eclean-pkg should have a flag to keep only one copy of...
Status: RESOLVED FIXED
Alias: None
Product: Portage Development
Classification: Unclassified
Component: Enhancement/Feature Requests (show other bugs)
Hardware: All Linux
: Normal normal (vote)
Assignee: Portage team
URL:
Whiteboard:
Keywords: PATCH
Depends on:
Blocks:
 
Reported: 2020-06-08 20:33 UTC by Rick Farina (Zero_Chaos)
Modified: 2022-05-31 16:48 UTC (History)
2 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Rick Farina (Zero_Chaos) gentoo-dev 2020-06-08 20:33:57 UTC
with FEATURES=binpkg-multi-instance portage can create many binpkgs for the same ebuild, possibly with different use flags, possibly with the same use flags.  eclean-pkg should, optionally or by default, clean all packages with the same use flag settings leaving only the most recent mtime package.
Comment 1 Zac Medico gentoo-dev 2020-06-09 02:17:26 UTC
People like me use `eclean-pkg -d` to prune any binary packages that are not currently installed, which fits the description of this bug as it's currently worded.

Since you want to keep the most recent build for each unique USE flag combination, I suppose you could call this option --unique-use.
Comment 2 Rick Farina (Zero_Chaos) gentoo-dev 2020-06-09 17:35:54 UTC
-d is a bit too distructive.  The purpose of binpkg-multi-instance is to be able to make binpkgs with different use settings, as such, they can't all be installed.

And yes, --unique-use would be fantastic
Comment 3 Rick Farina (Zero_Chaos) gentoo-dev 2020-06-14 02:45:07 UTC
Just for anyone reading this who hasn't seen all the irc conversations, let me explain a tiny bit better.

FEATURES=binpkg-multi-instance can make N binpkgs for an ebuild.  The general purpose of this feature (as I understand it) is to allow making binpkgs for different use flag combinations.  Unfortunately, if you ask, portage will just keep making new binpkgs with the *same* use flags (emerge --usepkg=n --buildpkg=y).  While this whole idea seems stupid, it happens quite a lot when dependencies are imperfect and some linkage level breakage is detected, possibly with revdep-rebuild or other similar tools.  Similarly, "smart-live-rebuild" would have the same issue, as it doesn't make sense to use a binpkg made from an old SCM revision when a new revision is detected.  As such, binpkgs with duplicate use flags are generated and there is no sane way to clean them until the ebuild is shadowed by a new version (for example).  In the case of smart-live-rebuild, I suspect the binpkgs will simply build up until the end of time.

"eclean-pkg" by itself only cleans up things which are no longer installable, but will happily leave 50 identical binpkgs in place.  "eclean-pkg --changed-deps" might clean up a few of our problematic packages, but certainly can't handle the SCM ebuild case.  Entertainingly, "--buildpkg-exclude */*-9999" doesn't appear to be valid, so if a user has binpkgs turned on by default, there is no way to prevent binpkgs from being generated when installing a SCM ebuild.  As I mentioned early, "eclean-pkg -d" is both too destructive (removes packages built with use flags not matching the user's current settings) and not destructive enough (if there are 50 binpkgs matching my version and use flag settings I don't believe they will be cleaned).

For the sake of completeness, I think it's important to note that portage will install the matching package with the latest mtime.  This means that if I have 50 binpkgs that match the requested settings, portage is already ignoring 49 of them, and simply going with the newest one.  While this behavior is both sane and desirable, the wasted disk space and parsing time of the Packages file may not go unnoticed by some.  I, for example, seem to have messed up the deps of net-wireless/uhd at some point, and ended up with 20 binpkgs as a result, each of them 200mb.  It would be awesome for a way to clean up the packages that portage will never under any circumstances use.
Comment 4 Yixun Lan archtester gentoo-dev 2022-05-12 08:05:39 UTC
I second the idea of "--unique-use", the use case I can think of 

with FEATURES=binpkg-multi-instance enabled, we could have binpkgs merged from two different profiles, so it will let many binpkgs co-eixst, which all of them are built from one single package but with different USE combinations. and for binpkgs with exactly the same USE flags, to keep most latest version is kind of sane
Comment 5 Yixun Lan archtester gentoo-dev 2022-05-25 08:22:39 UTC
https://github.com/gentoo/gentoolkit/pull/20

hi, i've created a PR for this, but I'd highly appreciate if someone willing to help to review or improve the code, since my python skill is really terrible..
thanks all
Comment 6 Yixun Lan archtester gentoo-dev 2022-05-28 09:26:14 UTC
note, current logic would like this: 
  --unique-use can be combined with --changed-deps, but will disabled if passing --deep
Comment 7 Larry the Git Cow gentoo-dev 2022-05-28 15:28:01 UTC
The bug has been closed via the following commit(s):

https://gitweb.gentoo.org/proj/gentoolkit.git/commit/?id=2fffbd450df2443bbd671f7ec760051ce3c930e9

commit 2fffbd450df2443bbd671f7ec760051ce3c930e9
Author:     Yixun Lan <dlan@gentoo.org>
AuthorDate: 2022-05-25 08:08:09 +0000
Commit:     Yixun Lan <dlan@gentoo.org>
CommitDate: 2022-05-28 15:24:32 +0000

    implement --unique-use option for eclean-pkg
    
    this will help to clean duplicated binpkg which has exact same USEs flags enabled,
    and it will choose the more recent version according to BUILD_TIME by default
    
    Closes: https://bugs.gentoo.org/727576
    Closes: https://github.com/gentoo/gentoolkit/pull/20
    Signed-off-by: Yixun Lan <dlan@gentoo.org>

 pym/gentoolkit/eclean/cli.py    | 14 +++++++++++++-
 pym/gentoolkit/eclean/search.py | 26 ++++++++++++++++++++++++++
 2 files changed, 39 insertions(+), 1 deletion(-)
Comment 8 Rick Farina (Zero_Chaos) gentoo-dev 2022-05-31 16:48:23 UTC
Thanks dlan!