| Summary: | sys-apps/portage-2.2_rc62: problem with tar.lzma archives | ||
|---|---|---|---|
| Product: | Gentoo Linux | Reporter: | Rafał Mużyło <galtgendo> |
| Component: | Current packages | Assignee: | Portage team <dev-portage> |
| Status: | RESOLVED UPSTREAM | ||
| Severity: | normal | CC: | niogic |
| Priority: | High | ||
| Version: | unspecified | ||
| Hardware: | All | ||
| OS: | Linux | ||
| Whiteboard: | |||
| Package list: | Runtime testing required: | --- | |
| Attachments: |
patch the libtool-2.2.6b ebuild to create $T/lzma.strace
strace with current xz-utils git |
||
|
Description
Rafał Mużyło
2010-02-17 16:06:59 UTC
Duplicate of bug 293580? Out of memory. While I do have very little memory, it's very unlikely this is the same problem - message is clearly different. The archives unpack correctly with default settings. It works on the commandline - I mean the version from '_unpack_tar lzma' (lzma -dc "$srcdir$x" | tar xof -), it only fails, while invoked by portage. (In reply to comment #2) > '_unpack_tar lzma' (lzma -dc "$srcdir$x" | tar xof -), > it only fails, while invoked by portage. You need to check both elements of ${PIPESTATUS[@]} or else you can't be sure. It unpacks successfully for me, so I suspect that you are experiencing bug 293580. No luck, XZ_OPTS doesn't affect if at all.
Now, there's a funny thing: while tar with -J works,
lzma -dc -- "${s}" | tar xof - || die
fails.
...makes no sense to me.
What exactly does that PIPESTATUS var reports (and how) ?
Cause on the commandline i.e.
lzma -dc -- netpbm-10.48.00.tar.lzma | tar xof - ; echo $PIPESTATUS
prints '0' and unpacks correctly.
a problem with shell ?
(In reply to comment #4) > What exactly does that PIPESTATUS var reports (and how) ? > > Cause on the commandline i.e. > lzma -dc -- netpbm-10.48.00.tar.lzma | tar xof - ; echo $PIPESTATUS > > prints '0' and unpacks correctly. PIPESTATUS is an internal bash array that contains the status of each process in the previous pipe. You have to echo ${PIPESTATUS[@]} or else you won't see all elements of the array. Well, changing in the last example $PIPESTATUS to ${PIPESTATUS[@]}
changes output to
'0 0'
That's still a correct unpack.
Well, that's exactly what portage does. So, I guess it's failing when portage does it because then there's more memory load? While that seems possible, it doesn't work even for XZ_OPT="--memory=64M". While this system does quite a bit of swapping, there should be enough memory to unpack an archive. And there's still that strange error for lzma. "Unexpected end of input" ? (In reply to comment #8) > And there's still that strange error for lzma. > "Unexpected end of input" ? That suggests that the lzma program is failing to produce all of the input that tar is expecting. I you use --debug then it should give more info. Here's the relevant part when I run unpack with --debug: + lzma -dc /var/tmp/portage/sys-devel/libtool-2.2.6b/distdir/libtool-2.2.6b.tar.lzma + assert 'failure unpacking libtool-2.2.6b.tar.lzma' + local x 'pipestatus=0 0' + for x in '$pipestatus' + [[ 0 -eq 0 ]] + for x in '$pipestatus' + [[ 0 -eq 0 ]] For me that part looks:
+ case "${x##*.}" in
+ _unpack_tar lzma
+ '[' tar == tar ']'
+ tar xof -
+ lzma -dc /var/tmp/portage/sys-devel/libtool-2.2.6b/distdir/libtool-2.2.6b.tar.lzma
lzma: /var/tmp/portage/sys-devel/libtool-2.2.6b/distdir/libtool-2.2.6b.tar.lzma: Unexpected end of input
tar: To nie wygląda jak archiwum tar
tar: Zakończenie w stanie błędu z powodu uprzednich błędów
+ assert 'failure unpacking libtool-2.2.6b.tar.lzma'
+ local x 'pipestatus=1 2'
(testing on libtool ebuild seemed most simple)
Maybe update/rebuild xz-utils in case that helps. It's 9999 and was rebuilt yesterday. And if that should be memory management, then why does 'tar xoJf' suceed ? It works here with xz-utils-4.999.9_beta. Maybe there's a bug in 9999. It would have to be strange one, as both commandline and texlive 2009 ebuilds (so tar.xz) work. Though now that you mention it, portage started to feel a bit sluggish lately, as if something with memory management was wrong (as mentioned, I do rely on swap quite a bit, mainly due to firefox). However, after I close firefox, I've got a bit over 200MB free, so I think more than enough and it still fails. So OK, with beta it works fine, however, given that it works just fine on the command line, I doubt upstream will be able to fix it without a testcase, especially that AFAIK it consists of a single person. *** Bug 307929 has been marked as a duplicate of this bug. *** (In reply to comment #18) > *** Bug 307929 has been marked as a duplicate of this bug. *** > Happy to see, it's not just my machine. However, the other reporter is asked for info about his machine. > the other reporter is asked for info
> about his machine.
Hi it's me.
Well it's a ~x86. Just updated xz-utils has been broken for at least 2 weeks for me. Latest git still broken.
Or I think I read it's portage problem.
It looks like we're going to have to do some debugging in xz-utils. Maybe we'll get a clue if we strace it when it's failing. (In reply to comment #21) > It looks like we're going to have to do some debugging in xz-utils. Maybe we'll > get a clue if we strace it when it's failing. > We need a testing custom ebuild with customized unpack stage. Created attachment 222297 [details, diff]
patch the libtool-2.2.6b ebuild to create $T/lzma.strace
Please try this patch and attach $T/lzma.strace from a failed unpack.
Created attachment 222313 [details]
strace with current xz-utils git
It did a single read from libtool-2.2.6b.tar.lzma which returned a full 8192 byte buffer, then without trying to read any more data it showed this error before exiting with status 1: lzma: /var/tmp/portage/sys-devel/libtool-2.2.6b/distdir/libtool-2.2.6b.tar.lzma: Unexpected end of input After IRCing with upstream a bit, it's fixed in the git now. (In reply to comment #26) > After IRCing with upstream a bit, it's fixed in the git now. Great, I guess people who have this issue can just rebuild xz-utils-9999 in order to get the fix. |
