We need a way to remove invalid binpkgs. I took a poll in #gentoo-portage and people seemed to prefer eclean-pkg doing this rather than an emaint subcommand. eclean-pkg should likely do this by default, with a --no-invalid or similar to allow keeping them behind for investigation.
https://github.com/gentoo/gentoolkit/pull/28 On it
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/proj/portage.git/commit/?id=71daef3ac877329a0479a72ba333a9c801a36bf3 commit 71daef3ac877329a0479a72ba333a9c801a36bf3 Author: Siddhanth Rathod <xsiddhanthrathod@gmail.com> AuthorDate: 2023-03-23 16:28:14 +0000 Commit: Sam James <sam@gentoo.org> CommitDate: 2023-08-21 05:15:05 +0000 bintree: Add invalid_paths member to API Add object invalid_paths for new eclean feature to deal with invalid binpkgs. This is needed for gentoolkit's eclean to handle cleaning up invalid binpkgs. gentoolkit side PR: https://github.com/gentoo/gentoolkit/pull/28 Bug: https://bugs.gentoo.org/900224 Signed-off-by: Siddhanth Rathod <xsiddhanthrathod@gmail.com> Closes: https://github.com/gentoo/portage/pull/1016 Signed-off-by: Sam James <sam@gentoo.org> NEWS | 7 +++++++ lib/portage/dbapi/bintree.py | 4 ++++ lib/portage/tests/dbapi/test_bintree.py | 1 + 3 files changed, 12 insertions(+)
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/proj/gentoolkit.git/commit/?id=a16d0d4fbfb4614832c4b682b41284a9050af29f commit a16d0d4fbfb4614832c4b682b41284a9050af29f Author: Siddhanth Rathod <xsiddhanthrathod@gmail.com> AuthorDate: 2023-03-23 12:14:36 +0000 Commit: Sam James <sam@gentoo.org> CommitDate: 2023-08-21 05:16:18 +0000 eclean-pkg: deal with invalid binpkgs Changes required in portage -> https://github.com/gentoo/portage/pull/1016 Bug: https://bugs.gentoo.org/900224 Signed-off-by: Siddhanth Rathod <xsiddhanthrathod@gmail.com> Signed-off-by: Sam James <sam@gentoo.org> pym/gentoolkit/eclean/cli.py | 47 +++++++++++++++++++++++++++++++++++++---- pym/gentoolkit/eclean/output.py | 14 ++++++++++++ pym/gentoolkit/eclean/search.py | 9 ++++++-- 3 files changed, 64 insertions(+), 6 deletions(-)
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=93862e8ea357f71a7fce72167329d6d4a43f5b64 commit 93862e8ea357f71a7fce72167329d6d4a43f5b64 Author: Sam James <sam@gentoo.org> AuthorDate: 2023-08-21 05:22:51 +0000 Commit: Sam James <sam@gentoo.org> CommitDate: 2023-08-21 05:22:51 +0000 app-portage/gentoolkit: live needs >=portage-3.0.52 Needed for new bintrees API member for eclean. Bug: https://bugs.gentoo.org/900224 Signed-off-by: Sam James <sam@gentoo.org> app-portage/gentoolkit/gentoolkit-9999.ebuild | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-)
Not sure if it was the recent gentoolkit commits or the recent portage ones, but two bad things happen (I am running -9999 of both just built before making this comment): 1. eclean-dist --deep bails with error: Traceback (most recent call last): File "/usr/lib/python-exec/python3.11/eclean-dist", line 41, in <module> main() File "/usr/lib/python3.11/site-packages/gentoolkit/eclean/cli.py", line 714, in main doAction(action, options, exclude=exclude, output=output) File "/usr/lib/python3.11/site-packages/gentoolkit/eclean/cli.py", line 622, in doAction if invalids and options["clean-invalids"]: ^^^^^^^^ UnboundLocalError: cannot access local variable 'invalids' where it is not associated with a value 2. eclean-dist without any flags seems to burn CPU and IO but never finishes (I let it run around 10 minutes).
Hello Joe Kappus, I would like to acknowledge that the issue at hand was indeed caused by my commit. I want to sincerely apologize for any inconvenience this may have caused. To rectify the situation, I have submitted a pull request that addresses the problem.
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/proj/gentoolkit.git/commit/?id=204b1abca2c0cbbdad188c7fd4b84452c2f630a0 commit 204b1abca2c0cbbdad188c7fd4b84452c2f630a0 Author: Siddhanth Rathod <xsiddhanthrathod@gmail.com> AuthorDate: 2023-08-22 13:55:48 +0000 Commit: Sam James <sam@gentoo.org> CommitDate: 2023-08-22 20:34:30 +0000 eclean-pkg: Limit invalid search to 'packages' action Bug: https://bugs.gentoo.org/900224 Fixes: a16d0d4fbfb4614832c4b682b41284a9050af29f Signed-off-by: Siddhanth Rathod <xsiddhanthrathod@gmail.com> Closes: https://github.com/gentoo/gentoolkit/pull/30 Signed-off-by: Sam James <sam@gentoo.org> pym/gentoolkit/eclean/cli.py | 46 +++++++++++++++++++++++--------------------- 1 file changed, 24 insertions(+), 22 deletions(-)
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/proj/gentoolkit.git/commit/?id=6b17633d0d74cb8fa0eeb46c8ee4f4f05d0e38b2 commit 6b17633d0d74cb8fa0eeb46c8ee4f4f05d0e38b2 Author: Sam James <sam@gentoo.org> AuthorDate: 2023-08-22 20:43:25 +0000 Commit: Sam James <sam@gentoo.org> CommitDate: 2023-08-22 20:49:55 +0000 eclean-pkg: add --clean-invalid to help, fix parsing * Rename --clean-invalids -> --clean-invalid * Add --clean-invalid to help * Make it default-off for now to match the option name (rather than the help saying --no-clean-invalids). (I could go either way on what the actual default should be, but this is more conservative for now.) * Fix typo in option name. * Make it a package specific arg, not a global one. Bug: https://bugs.gentoo.org/900224 Fixes: 204b1abca2c0cbbdad188c7fd4b84452c2f630a0 Fixes: a16d0d4fbfb4614832c4b682b41284a9050af29f Signed-off-by: Sam James <sam@gentoo.org> pym/gentoolkit/eclean/cli.py | 19 ++++++++++++------- 1 file changed, 12 insertions(+), 7 deletions(-)
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/proj/gentoolkit.git/commit/?id=d40b5009c1f73730ad4d68169b1302393cdce8cf commit d40b5009c1f73730ad4d68169b1302393cdce8cf Author: Sam James <sam@gentoo.org> AuthorDate: 2023-08-22 20:52:35 +0000 Commit: Sam James <sam@gentoo.org> CommitDate: 2023-08-22 20:52:35 +0000 eclean-pkg: fix --help alignment, mention shortopt for clean-invalid Bug: https://bugs.gentoo.org/900224 Signed-off-by: Sam James <sam@gentoo.org> pym/gentoolkit/eclean/cli.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
I would strongly recommend cleaning invalids on any cleaning action. There is no reason to keep these packages, at best portage is going to ignore them, at worst it's going to crash. In my experience most of the files are either 0 bytes or no longer valid due to a pkgmove.
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/proj/gentoolkit.git/commit/?id=9ea9cfe27f3debe4f97f3b04da6681fc3ae11b88 commit 9ea9cfe27f3debe4f97f3b04da6681fc3ae11b88 Author: Siddhanth Rathod <xsiddhanthrathod@gmail.com> AuthorDate: 2023-08-24 15:41:35 +0000 Commit: Sam James <sam@gentoo.org> CommitDate: 2023-08-24 19:52:20 +0000 eclean: Make cleaning invalid default Bug: https://bugs.gentoo.org/900224 Signed-off-by: Siddhanth Rathod <xsiddhanthrathod@gmail.com> Closes: https://github.com/gentoo/gentoolkit/pull/31 Signed-off-by: Sam James <sam@gentoo.org> pym/gentoolkit/eclean/cli.py | 15 ++++++++------- 1 file changed, 8 insertions(+), 7 deletions(-)
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=9a9621d863c8829b54cdd76124c94f83ddd006c5 commit 9a9621d863c8829b54cdd76124c94f83ddd006c5 Author: Sam James <sam@gentoo.org> AuthorDate: 2023-08-27 00:08:30 +0000 Commit: Sam James <sam@gentoo.org> CommitDate: 2023-08-27 00:08:30 +0000 app-portage/gentoolkit: update comment wrt portage version Bug: https://bugs.gentoo.org/900224 Signed-off-by: Sam James <sam@gentoo.org> app-portage/gentoolkit/gentoolkit-9999.ebuild | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/proj/portage.git/commit/?id=cec349d7cfcda634e3f76cd3ca0c1a89bc46f414 commit cec349d7cfcda634e3f76cd3ca0c1a89bc46f414 Author: hypersyd <70613804+siddhanthrathod@users.noreply.github.com> AuthorDate: 2023-09-09 13:51:31 +0000 Commit: Sam James <sam@gentoo.org> CommitDate: 2023-09-26 21:09:28 +0000 bintree: populate invalid_paths list for eclean-pkg Enable eclean-pkg to handle incomplete merges from binhost. It's an extension for gentoolkit's eclean-pkg clean invalids feature to deal with binpkgs created by the new binhost without any tar header or format due to incomplete merges. This results in gpkg.py --> class gpkg --> method _get_tar_format returning null which raises PortagePackageException and SignatureException in binpkg.py --> Class binarytree. Add an input for invalid paths api is being added so the eclean-pkg can clean it up. Invalids feature: https://github.com/gentoo/gentoolkit/commit/a16d0d4fbfb4614832c4b682b41284a9050af29f, https://github.com/gentoo/portage/commit/71daef3ac877329a0479a72ba333a9c801a36bf3 Bug: https://bugs.gentoo.org/900224 Signed-off-by: Siddhanth Rathod <xsiddhanthrathod@gmail.com> Closes: https://github.com/gentoo/portage/pull/1091 Signed-off-by: Sam James <sam@gentoo.org> lib/portage/dbapi/bintree.py | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-)
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/proj/gentoolkit.git/commit/?id=e49cc3493ab77678a740bf8a27cf22e77e061409 commit e49cc3493ab77678a740bf8a27cf22e77e061409 Author: Siddhanth Rathod <xsiddhanthrathod@gmail.com> AuthorDate: 2023-09-28 18:54:44 +0000 Commit: Sam James <sam@gentoo.org> CommitDate: 2023-09-29 07:20:18 +0000 eclean-pkg: fix compatibility with FEATURES=pkgdir-index-trusted Portage recently enabled FEATURES=pkgdir-index-trusted by default which breaks the eclean-invalids feature. Force reindexing after cleaning up to fix this. See-also: https://github.com/gentoo/portage/commit/3bc7bfef51d88fd716e882d3931f2873e3493e01 Bug: https://bugs.gentoo.org/889300 Bug: https://bugs.gentoo.org/900224 Signed-off-by: Siddhanth Rathod <xsiddhanthrathod@gmail.com> Closes: https://github.com/gentoo/gentoolkit/pull/34 Signed-off-by: Sam James <sam@gentoo.org> pym/gentoolkit/eclean/search.py | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-)
The bug has been closed via the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=9f9211a35abef13079f93a96f57f3a96083c69a7 commit 9f9211a35abef13079f93a96f57f3a96083c69a7 Author: Sam James <sam@gentoo.org> AuthorDate: 2023-10-03 15:38:28 +0000 Commit: Sam James <sam@gentoo.org> CommitDate: 2023-10-03 15:38:41 +0000 sys-apps/portage: add 3.0.52 Closes: https://bugs.gentoo.org/704866 Closes: https://bugs.gentoo.org/877793 Closes: https://bugs.gentoo.org/889300 Closes: https://bugs.gentoo.org/900224 Closes: https://bugs.gentoo.org/912676 Closes: https://bugs.gentoo.org/912808 Closes: https://bugs.gentoo.org/913070 Closes: https://bugs.gentoo.org/913103 Closes: https://bugs.gentoo.org/914159 Closes: https://bugs.gentoo.org/915054 Closes: https://bugs.gentoo.org/915119 Signed-off-by: Sam James <sam@gentoo.org> sys-apps/portage/Manifest | 1 + sys-apps/portage/portage-3.0.52.ebuild | 235 +++++++++++++++++++++++++++++++++ 2 files changed, 236 insertions(+)
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=c162ce62447fec711a28ecf1434879256c8aea6a commit c162ce62447fec711a28ecf1434879256c8aea6a Author: Sam James <sam@gentoo.org> AuthorDate: 2023-10-03 15:43:51 +0000 Commit: Sam James <sam@gentoo.org> CommitDate: 2023-10-03 15:44:35 +0000 app-portage/gentoolkit: add 0.6.2 Bug: https://bugs.gentoo.org/900224 Closes: https://bugs.gentoo.org/369581 Closes: https://bugs.gentoo.org/382009 Closes: https://bugs.gentoo.org/576224 Closes: https://bugs.gentoo.org/709816 Closes: https://bugs.gentoo.org/861116 Closes: https://bugs.gentoo.org/877519 Closes: https://bugs.gentoo.org/902551 Signed-off-by: Sam James <sam@gentoo.org> app-portage/gentoolkit/Manifest | 1 + app-portage/gentoolkit/gentoolkit-0.6.2.ebuild | 71 ++++++++++++++++++++++++++ 2 files changed, 72 insertions(+)
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/proj/gentoolkit.git/commit/?id=a12d24f99d03c3c4975d84d63083449cd8487ef9 commit a12d24f99d03c3c4975d84d63083449cd8487ef9 Author: Siddhanth Rathod <xsiddhanthrathod@gmail.com> AuthorDate: 2023-10-05 13:34:59 +0000 Commit: Sam James <sam@gentoo.org> CommitDate: 2023-10-08 03:27:53 +0000 eclean: pkg: suppress invalid binary error [sam: Note that this needs https://github.com/gentoo/portage/pull/1123 on the Portage side.] Bug: https://bugs.gentoo.org/900224 Signed-off-by: Siddhanth Rathod <xsiddhanthrathod@gmail.com> Closes: https://github.com/gentoo/gentoolkit/pull/35 Signed-off-by: Sam James <sam@gentoo.org> pym/gentoolkit/eclean/search.py | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/proj/portage.git/commit/?id=c53d046c9629d5c3a7841aee4e92ae38c0691e69 commit c53d046c9629d5c3a7841aee4e92ae38c0691e69 Author: Siddhanth Rathod <xsiddhanthrathod@gmail.com> AuthorDate: 2023-10-05 13:22:52 +0000 Commit: Sam James <sam@gentoo.org> CommitDate: 2023-10-08 03:29:46 +0000 dbapi: bintree: introduce invalid_errors Introduce 'invalid_errors' var to enable suppression of invalid binary error, for use by gentoolkit. [sam: See gentoolkit side at https://github.com/gentoo/gentoolkit/pull/35]. Bug: https://bugs.gentoo.org/900224 Signed-off-by: Siddhanth Rathod <xsiddhanthrathod@gmail.com> Closes: https://github.com/gentoo/portage/pull/1123 Signed-off-by: Sam James <sam@gentoo.org> lib/portage/dbapi/bintree.py | 60 +++++++++++++++++++-------------- lib/portage/tests/dbapi/test_bintree.py | 6 ++-- 2 files changed, 39 insertions(+), 27 deletions(-)
The bug has been closed via the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=3500483f75789c36e379c1b6546a09df7c11e2b1 commit 3500483f75789c36e379c1b6546a09df7c11e2b1 Author: Sam James <sam@gentoo.org> AuthorDate: 2023-10-20 00:49:33 +0000 Commit: Sam James <sam@gentoo.org> CommitDate: 2023-10-20 00:51:00 +0000 sys-apps/portage: add 3.0.53 Closes: https://bugs.gentoo.org/915120 Closes: https://bugs.gentoo.org/821529 Closes: https://bugs.gentoo.org/914441 Closes: https://bugs.gentoo.org/914722 Closes: https://bugs.gentoo.org/914873 Closes: https://bugs.gentoo.org/915099 Closes: https://bugs.gentoo.org/915123 Closes: https://bugs.gentoo.org/915128 Closes: https://bugs.gentoo.org/915136 Closes: https://bugs.gentoo.org/915330 Closes: https://bugs.gentoo.org/915494 Closes: https://bugs.gentoo.org/915834 Closes: https://bugs.gentoo.org/915903 Closes: https://bugs.gentoo.org/900224 Signed-off-by: Sam James <sam@gentoo.org> sys-apps/portage/Manifest | 1 + sys-apps/portage/portage-3.0.53.ebuild | 238 +++++++++++++++++++++++++++++++++ 2 files changed, 239 insertions(+)
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=122ee4d7e65a6a1ba60a508a99f4d93632bda5dd commit 122ee4d7e65a6a1ba60a508a99f4d93632bda5dd Author: Sam James <sam@gentoo.org> AuthorDate: 2023-10-20 00:57:42 +0000 Commit: Sam James <sam@gentoo.org> CommitDate: 2023-10-20 00:58:01 +0000 app-portage/gentoolkit: add 0.6.3 Bug: https://bugs.gentoo.org/900224 Closes: https://bugs.gentoo.org/909888 Signed-off-by: Sam James <sam@gentoo.org> app-portage/gentoolkit/Manifest | 1 + app-portage/gentoolkit/gentoolkit-0.6.3.ebuild | 121 +++++++++++++++++++++++++ app-portage/gentoolkit/gentoolkit-9999.ebuild | 6 +- 3 files changed, 125 insertions(+), 3 deletions(-)
