My emerge --sync is consistently crashing since my last update: emerge --sync >>> Syncing repository 'gentoo' into '/usr/portage'... Do you want to sync your ebuild repository with the mirror at rsync://89.238.71.6/gentoo-portage? [Yes/No] >>> Starting rsync with rsync://89.238.71.6/gentoo-portage... >>> Checking server timestamp ... Welcome to turnstone.gentoo.org / rsync.gentoo.org Server Address : 89.238.71.6, 2a00:1828:a00d:ffff::6 Contact Name : mirror-admin@gentoo.org Hardware : 16 x Intel(R) Xeon(R) CPU E5530 @ 2.40GHz, 24152MB RAM Sponsor : Manitu GmbH, St. Wendel, Germany Please note: common gentoo-netiquette says you should not sync more than once a day. Users who abuse the rsync.gentoo.org rotation may be added to a temporary ban list. MOTD autogenerated by update-rsync-motd on Tue Oct 11 08:08:51 PM UTC 2022 receiving incremental file list timestamp.chk 32 100% 31.25kB/s 0:00:00 (xfr#1, to-chk=0/1) Number of files: 1 (reg: 1) Number of created files: 0 Number of deleted files: 0 Number of regular files transferred: 1 Total file size: 32 bytes Total transferred file size: 32 bytes Literal data: 32 bytes Matched data: 0 bytes File list size: 42 File list generation time: 0.001 seconds File list transfer time: 0.000 seconds Total bytes sent: 104 Total bytes received: 133 sent 104 bytes received 133 bytes 474.00 bytes/sec total size is 32 speedup is 0.14 >>> >>> Timestamps on the server and in the local repository are the same. >>> Cancelling all further sync action. You are already up to date. >>> >>> In order to force sync, remove '/usr/portage/metadata/timestamp.chk'. >>> === Sync completed for gentoo Performing Global Updates (Could take a couple of minutes if you have a lot of binary packages.) .='update pass' *='binary update' #='/var/db update' @='/var/db move' s='/var/db SLOT move' %='binary move' S='binary SLOT move' p='update /etc/portage/package.*' /usr/portage/profiles/updates/1Q-2023..... Traceback (most recent call last): File "/usr/lib/python3.9/site-packages/portage/xpak.py", line 441, in scan get_binpkg_format(self.file, check_file=True) File "/usr/lib/python3.9/site-packages/portage/binpkg.py", line 48, in get_binpkg_format if "gpkg-1" in (os.path.basename(f) for f in gpkg_tar.getnames()): File "/usr/lib/python3.9/tarfile.py", line 1816, in getnames return [tarinfo.name for tarinfo in self.getmembers()] File "/usr/lib/python3.9/tarfile.py", line 1808, in getmembers self._load() # all members, we first have to File "/usr/lib/python3.9/tarfile.py", line 2406, in _load tarinfo = self.next() File "/usr/lib/python3.9/tarfile.py", line 2328, in next self.fileobj.seek(self.offset - 1) File "/usr/lib/python3.9/bz2.py", line 275, in seek return self._buffer.seek(offset, whence) File "/usr/lib/python3.9/_compression.py", line 143, in seek data = self.read(min(io.DEFAULT_BUFFER_SIZE, offset)) File "/usr/lib/python3.9/_compression.py", line 99, in read raise EOFError("Compressed file ended before the " EOFError: Compressed file ended before the end-of-stream marker was reached During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/lib/python-exec/python3.9/emerge", line 58, in <module> retval = emerge_main() File "/usr/lib/python3.9/site-packages/_emerge/main.py", line 1294, in emerge_main return run_action(emerge_config) File "/usr/lib/python3.9/site-packages/_emerge/actions.py", line 3908, in run_action return action_sync(emerge_config) File "/usr/lib/python3.9/site-packages/_emerge/actions.py", line 2461, in action_sync success, msgs = syncer.auto_sync(options=options) File "/usr/lib/python3.9/site-packages/portage/emaint/modules/sync/sync.py", line 90, in auto_sync return self._sync(repos, return_messages, emaint_opts=options) File "/usr/lib/python3.9/site-packages/portage/emaint/modules/sync/sync.py", line 262, in _sync self._do_pkg_moves() File "/usr/lib/python3.9/site-packages/portage/emaint/modules/sync/sync.py", line 273, in _do_pkg_moves if self.emerge_config.opts.get("--package-moves") != "n" and _global_updates( File "/usr/lib/python3.9/site-packages/portage/_global_updates.py", line 41, in _global_updates return _do_global_updates( File "/usr/lib/python3.9/site-packages/portage/_global_updates.py", line 133, in _do_global_updates bindb.bintree.populate() File "/usr/lib/python3.9/site-packages/portage/dbapi/bintree.py", line 826, in populate update_pkgindex = self._populate_local( File "/usr/lib/python3.9/site-packages/portage/dbapi/bintree.py", line 1037, in _populate_local pkg_metadata = self._read_metadata( File "/usr/lib/python3.9/site-packages/portage/dbapi/bintree.py", line 1720, in _read_metadata binpkg_metadata = portage.xpak.tbz2(filename).get_data() File "/usr/lib/python3.9/site-packages/portage/xpak.py", line 552, in get_data if not self.scan(): File "/usr/lib/python3.9/site-packages/portage/xpak.py", line 471, in scan get_binpkg_format(self.file, check_file=True) File "/usr/lib/python3.9/site-packages/portage/binpkg.py", line 48, in get_binpkg_format if "gpkg-1" in (os.path.basename(f) for f in gpkg_tar.getnames()): File "/usr/lib/python3.9/tarfile.py", line 1816, in getnames return [tarinfo.name for tarinfo in self.getmembers()] File "/usr/lib/python3.9/tarfile.py", line 1808, in getmembers self._load() # all members, we first have to File "/usr/lib/python3.9/tarfile.py", line 2406, in _load tarinfo = self.next() File "/usr/lib/python3.9/tarfile.py", line 2328, in next self.fileobj.seek(self.offset - 1) File "/usr/lib/python3.9/bz2.py", line 275, in seek return self._buffer.seek(offset, whence) File "/usr/lib/python3.9/_compression.py", line 143, in seek data = self.read(min(io.DEFAULT_BUFFER_SIZE, offset)) File "/usr/lib/python3.9/_compression.py", line 99, in read raise EOFError("Compressed file ended before the " EOFError: Compressed file ended before the end-of-stream marker was Reproducible: Always Steps to Reproduce: 1. emerge --sync Actual Results: crashes with "EOFError: Compressed file ended before the end-of-stream marker was reached" Expected Results: syncs successfully NOTE: my portage tree /usr/portage resides on a squashfs, but if I understood correctly, emerge should be oblivious to this fact. NOTE also: I just updated a whole bunch of packages, but my system should be consistent.
Created attachment 850236 [details] emerge --info my emerge --info
Is there any chance the binary package is broken / incomplete?
Hi, what binary package are you talking about?
Oh I suppose you mean /usr/portage/profiles/updates/1Q-2023 # cat /usr/portage/profiles/updates/1Q-2023 move net-im/discord-bin net-im/discord move dev-python/python-recurring-ical-events dev-python/recurring-ical-events move dev-python/snappy dev-python/python-snappy move dev-python/pkgcraft-python dev-python/pkgcraft move dev-python/python-nbxmpp dev-python/nbxmpp Is there some way to do these moves by hand instead?
(In reply to igel from comment #4) > Oh I suppose you mean > /usr/portage/profiles/updates/1Q-2023 > > # cat /usr/portage/profiles/updates/1Q-2023 > move net-im/discord-bin net-im/discord > move dev-python/python-recurring-ical-events dev-python/recurring-ical-events > move dev-python/snappy dev-python/python-snappy > move dev-python/pkgcraft-python dev-python/pkgcraft > move dev-python/python-nbxmpp dev-python/nbxmpp > > Is there some way to do these moves by hand instead? So the way this works is basically: These are package moves (renames!) We need to 'move' the binary packages. The update code is trying to do this for you. Here its trying to open a binary package (to do the update); the binary package is compressed, and we are failing to decompress it. Its likely a few things need improvement here: - The error doesn't really tell you which binary package is busted; we should fix that in portage. - Its likely moving the binary package out of the binpkg repo will 'fix' your updates. - We sort of want to know why one of your binary packages is corrupted; mostly so we can try to prevent corruption; or do something else when packages are corrupted.
(In reply to igel from comment #3) > Hi, what binary package are you talking about? Are you using binpkgs/binary packages with Portage? Is there anything in /var/cache/binpkgs or /usr/portage/packages?
Yes, I'm using quickpkg (I think) to backup my enlightenment & efl before I update them usually. Also, I run thunderbird-bin cause I really don't want to compile that. # find /var/cache/binpkgs /var/cache/binpkgs /var/cache/binpkgs/Packages /var/cache/binpkgs/mail-client /var/cache/binpkgs/mail-client/thunderbird-bin-78.10.1.tbz2 /var/cache/binpkgs/qpkg.ZwPDxe /var/cache/binpkgs/qpkg.ZwPDxe/bin.tbz2 /var/cache/binpkgs/qpkg.ZwPDxe/filelist /var/cache/binpkgs/dev-libs /var/cache/binpkgs/dev-libs/efl-1.25.1-r10.tbz2 /var/cache/binpkgs/x11-wm /var/cache/binpkgs/x11-wm/enlightenment-0.24.2.tbz2 A-ha! # bzip2 --test /var/cache/binpkgs/qpkg.ZwPDxe/bin.tbz2 bzip2: /var/cache/binpkgs/qpkg.ZwPDxe/bin.tbz2: file ends unexpectedly You can use the `bzip2recover' program to attempt to recover data from undamaged sections of corrupted files. I have no idea what this "bin.tbz2" is and why it's there, but the "filelist" file shows a whole lot of stuff related to enlightenment oO I suspect that removing this ominous "qpkg.ZwPDxe" will fix my issue and I'd do that unless you want me to do more debugging/testing before...
(In reply to igel from comment #7) > Yes, I'm using quickpkg (I think) to backup my enlightenment & efl before I > update them usually. Also, I run thunderbird-bin cause I really don't want > to compile that. > > # find /var/cache/binpkgs > /var/cache/binpkgs > /var/cache/binpkgs/Packages > /var/cache/binpkgs/mail-client > /var/cache/binpkgs/mail-client/thunderbird-bin-78.10.1.tbz2 > /var/cache/binpkgs/qpkg.ZwPDxe > /var/cache/binpkgs/qpkg.ZwPDxe/bin.tbz2 > /var/cache/binpkgs/qpkg.ZwPDxe/filelist > /var/cache/binpkgs/dev-libs > /var/cache/binpkgs/dev-libs/efl-1.25.1-r10.tbz2 > /var/cache/binpkgs/x11-wm > /var/cache/binpkgs/x11-wm/enlightenment-0.24.2.tbz2 > > A-ha! > # bzip2 --test /var/cache/binpkgs/qpkg.ZwPDxe/bin.tbz2 > bzip2: /var/cache/binpkgs/qpkg.ZwPDxe/bin.tbz2: file ends unexpectedly > > You can use the `bzip2recover' program to attempt to recover > data from undamaged sections of corrupted files. > > I have no idea what this "bin.tbz2" is and why it's there, but the > "filelist" file shows a whole lot of stuff related to enlightenment oO My guess is you ran `quickpkg <some-enlightenment-pkg>` and it crashed, or you hit ctrl+C or something. Quickpkg was in the middle of making your pkg and failed to cleanup properly. > > I suspect that removing this ominous "qpkg.ZwPDxe" will fix my issue and I'd > do that unless you want me to do more debugging/testing before... I still think portage should, if it encounters a problem doing a pkgmove, print the problematic package name ;) -A
(In reply to Alec Warner from comment #8) > (In reply to igel from comment #7) > > Yes, I'm using quickpkg (I think) to backup my enlightenment & efl before I > > update them usually. Also, I run thunderbird-bin cause I really don't want > > to compile that. > > > > # find /var/cache/binpkgs > > /var/cache/binpkgs > > /var/cache/binpkgs/Packages > > /var/cache/binpkgs/mail-client > > /var/cache/binpkgs/mail-client/thunderbird-bin-78.10.1.tbz2 > > /var/cache/binpkgs/qpkg.ZwPDxe > > /var/cache/binpkgs/qpkg.ZwPDxe/bin.tbz2 > > /var/cache/binpkgs/qpkg.ZwPDxe/filelist > > /var/cache/binpkgs/dev-libs > > /var/cache/binpkgs/dev-libs/efl-1.25.1-r10.tbz2 > > /var/cache/binpkgs/x11-wm > > /var/cache/binpkgs/x11-wm/enlightenment-0.24.2.tbz2 > > > > A-ha! > > # bzip2 --test /var/cache/binpkgs/qpkg.ZwPDxe/bin.tbz2 > > bzip2: /var/cache/binpkgs/qpkg.ZwPDxe/bin.tbz2: file ends unexpectedly > > > > You can use the `bzip2recover' program to attempt to recover > > data from undamaged sections of corrupted files. > > > > I have no idea what this "bin.tbz2" is and why it's there, but the > > "filelist" file shows a whole lot of stuff related to enlightenment oO > > My guess is you ran `quickpkg <some-enlightenment-pkg>` and it crashed, or > you hit ctrl+C or something. Quickpkg was in the middle of making your pkg > and failed to cleanup properly. > > > > > I suspect that removing this ominous "qpkg.ZwPDxe" will fix my issue and I'd > > do that unless you want me to do more debugging/testing before... > > I still think portage should, if it encounters a problem doing a pkgmove, > print the problematic package name ;) > > -A It is supposed to do that, but I didn't catch that category errors. Patch incoming.
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/proj/portage.git/commit/?id=6e1c42a89fe1ae7a5915c6eb8e3e27b47aecf93c commit 6e1c42a89fe1ae7a5915c6eb8e3e27b47aecf93c Author: Sheng Yu <syu.os@protonmail.com> AuthorDate: 2023-02-10 07:34:49 +0000 Commit: Sam James <sam@gentoo.org> CommitDate: 2023-02-17 01:23:14 +0000 Catch all Exceptions in binpkg identify We got many different exceptions here, so have to catch all. Bug: https://bugs.gentoo.org/893638 Signed-off-by: Sheng Yu <syu.os@protonmail.com> Signed-off-by: Sam James <sam@gentoo.org> lib/portage/binpkg.py | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-)
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=cd8ade10313d72cb0e3dd2229df02e0ea8681daa commit cd8ade10313d72cb0e3dd2229df02e0ea8681daa Author: Sam James <sam@gentoo.org> AuthorDate: 2023-02-26 22:01:01 +0000 Commit: Sam James <sam@gentoo.org> CommitDate: 2023-02-26 22:01:10 +0000 sys-apps/portage: add 3.0.45 Bug: https://bugs.gentoo.org/891001 Bug: https://bugs.gentoo.org/889330 Bug: https://bugs.gentoo.org/890777 Bug: https://bugs.gentoo.org/891391 Bug: https://bugs.gentoo.org/893638 Bug: https://bugs.gentoo.org/795825 Bug: https://bugs.gentoo.org/884869 Bug: https://bugs.gentoo.org/888585 Bug: https://bugs.gentoo.org/892651 Bug: https://bugs.gentoo.org/895526 Signed-off-by: Sam James <sam@gentoo.org> sys-apps/portage/Manifest | 1 + sys-apps/portage/portage-3.0.45.ebuild | 288 +++++++++++++++++++++++++++++++++ 2 files changed, 289 insertions(+)
