Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 710444 - sys-apps/portage-2.3.89 - can't extract xpaks made with BINPKG_COMPRESS="zstd" : zstd: error 11 : Parameter is out of bound
Summary: sys-apps/portage-2.3.89 - can't extract xpaks made with BINPKG_COMPRESS="zstd...
Status: RESOLVED FIXED
Alias: None
Product: Portage Development
Classification: Unclassified
Component: Binary packages support (show other bugs)
Hardware: All Linux
: Normal normal (vote)
Assignee: Portage team
URL:
Whiteboard:
Keywords: InVCS
Depends on: 634980
Blocks: 711148
  Show dependency tree
 
Reported: 2020-02-21 23:56 UTC by Enne Eziarc
Modified: 2020-11-13 09:13 UTC (History)
0 users

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Enne Eziarc 2020-02-21 23:56:28 UTC
On attempting to install a gcc binary from a crossdev host on i686, portage dumps me into the debugger with this backtrace and afterwards sits idle until I ctrl+c:

Traceback (most recent call last):
  File "/usr/lib/python3.8/asyncio/events.py", line 81, in _run
    self._context.run(self._callback, *self._args)
  File "/usr/lib/python3.8/site-packages/_emerge/AsynchronousTask.py", line 84, in wait
    self._wait_hook()
  File "/usr/lib/python3.8/site-packages/_emerge/AsynchronousTask.py", line 195, in _wait_hook
    self._exit_listener_stack.pop()(self)
  File "/usr/lib/python3.8/site-packages/_emerge/Binpkg.py", line 341, in _unpack_contents_exit
    unpack_contents.future.result()
  File "/usr/lib/python3.8/site-packages/portage/util/futures/compat_coroutine.py", line 111, in _next
    future = self._generator.send(previous.result())
  File "/usr/lib/python3.8/site-packages/portage/dbapi/bintree.py", line 289, in unpack_contents
    raise PortageException("Error Extracting '{}'".format(pkg_path))
portage.exception.PortageException: Error Extracting '/mnt/nfs/portage/pkgdir/i686-pc-linux-gnu/sys-devel/gcc/gcc-9.2.0-r4-1.xpak'
--Return--
> /usr/lib/python3.8/site-packages/portage/util/_eventloop/asyncio_event_loop.py(81)_internal_caller_exception_handler()->None
-> pdb.set_trace()
(Pdb)


The entire build.log:

>>> Extracting info
 * Package:    sys-devel/gcc-9.2.0-r4
 * Repository: gentoo
 * USE:        nls abi_x86_32 graphite pch vtv elibc_glibc cxx kernel_linux ssp nptl openmp lto x86 userland_GNU pgo sanitize pie
 * FEATURES:   compressdebug distcc network-sandbox preserve-libs sandbox userpriv usersandbox
>>> Extracting sys-devel/gcc-9.2.0-r4
zstd: error 11 : Parameter is out of bound
tar: This does not look like a tar archive
tar: Exiting with failure status due to previous errors
command 'zstd -d --long=31' failed with status 11


The "--long=31" is hardcoded in portage/util/compression_probe.py, and per zstd's manpage using it without specifying additional parameters is invalid. Simply deleting it from there allows emerge to complete.
Comment 1 Zac Medico gentoo-dev 2020-02-22 01:10:20 UTC
With dev-python/zstandard-0.13.0 I'm able to execute zstd -d --long=31 commands successfully.

Maybe you've found a zstandard bug there. Can you reproduce the error with any other files?
Comment 2 Zac Medico gentoo-dev 2020-02-22 01:11:15 UTC
Oh, maybe it's because you're on x86. I tested on amd64.
Comment 3 Zac Medico gentoo-dev 2020-03-14 22:34:07 UTC
The "zstd: error 11 : Parameter is out of bound" error is only triggered after upgrade to app-arch/zstd-1.4.4-r2 from app-arch/zstd-1.3.7-r1.
Comment 4 Zac Medico gentoo-dev 2020-03-14 22:45:50 UTC
Looks like the change was probably triggered by this commit from https://github.com/facebook/zstd/pull/1417:

https://github.com/facebook/zstd/commit/d7da3fc90ae31cd5397002150a44418917179420
Comment 5 Larry the Git Cow gentoo-dev 2020-03-14 23:40:29 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/proj/portage.git/commit/?id=07da257cfc80509c50104560b1e1508b9e585b98

commit 07da257cfc80509c50104560b1e1508b9e585b98
Author:     Zac Medico <zmedico@gentoo.org>
AuthorDate: 2020-03-14 23:18:55 +0000
Commit:     Zac Medico <zmedico@gentoo.org>
CommitDate: 2020-03-14 23:21:55 +0000

    compression_probe: omit zstd --long=31 on 32-bit arch (bug 710444)
    
    Omit the zstd --long=31 argument for decompression on 32-bit
    architectures, since the latest version of zstd will otherwise
    abort with an error on 32-bit architectures.
    
    Bug: https://bugs.gentoo.org/710444
    Bug: https://bugs.gentoo.org/634980
    Signed-off-by: Zac Medico <zmedico@gentoo.org>

 lib/portage/util/compression_probe.py | 10 ++++++++--
 1 file changed, 8 insertions(+), 2 deletions(-)
Comment 6 Larry the Git Cow gentoo-dev 2020-03-15 01:29:33 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=729dc030f6b318f84ca6336c7bd7dddaa7e48040

commit 729dc030f6b318f84ca6336c7bd7dddaa7e48040
Author:     Zac Medico <zmedico@gentoo.org>
AuthorDate: 2020-03-15 01:23:23 +0000
Commit:     Zac Medico <zmedico@gentoo.org>
CommitDate: 2020-03-15 01:29:25 +0000

    sys-apps/portage: Bump to version 2.3.94
    
     #692492 secure ebuild ${D} permissions
     #710444 omit zstd --long=31 for decompress on 32-bit arch
     #712298 respect emerge --deep=<depth> with --update
    
    Bug: https://bugs.gentoo.org/711148
    Bug: https://bugs.gentoo.org/692492
    Bug: https://bugs.gentoo.org/710444
    Bug: https://bugs.gentoo.org/712298
    Package-Manager: Portage-2.3.94, Repoman-2.3.20
    Signed-off-by: Zac Medico <zmedico@gentoo.org>

 sys-apps/portage/Manifest              |   1 +
 sys-apps/portage/portage-2.3.94.ebuild | 271 +++++++++++++++++++++++++++++++++
 2 files changed, 272 insertions(+)