Bugzilla Bug 144624

emerging fcdsl fails, as it appears to be trying to unpack a gzip archive
rather than the downloaded rpm file.

emerge fcdsl

Calculating dependencies... done!
>>> Emerging (1 of 1) net-dialup/fcdsl-2.6.37 to /
>>> checking ebuild checksums ;-)
>>> checking auxfile checksums ;-)
>>> checking miscfile checksums ;-)
>>> checking km_fcdsl-2.6-37.i586.rpm ;-)
>>> checking avm_fcdsl-2.6-37.i586.rpm ;-)
 * Determining the location of the kernel source code
 * Found kernel source directory:
 *     /usr/src/linux
 * Found sources for kernel version:
 *     2.6.16-hardened-r11
 * Checking for suitable kernel configuration options...                  [ ok
] *
 * You can control the modules which are built with the variable
 * FCDSL_CARDS which should contain a blank separated list
 * of a selection from the following cards:
 *    fcdsl2 fcdslsl fcdslusb fcdslslusb fcdslusb2 fcdsl fcdslusba
>>> Unpacking source...

gzip: stdin: not in gzip format
cpio: premature end of archive

!!! ERROR: net-dialup/fcdsl-2.6.37 failed.
Call stack:
  ebuild.sh, line 1539:   Called dyn_unpack
  ebuild.sh, line 711:   Called src_unpack
  fcdsl-2.6.37.ebuild, line 71:   Called die

!!! failed to unpack avm_fcdsl-2.6-37.i586.rpm file
!!! If you need support, post the topmost build error, and the call stack if
relevant.

this bug is known to us. on repetitive runs it fails sometimes and sometimes
not. If you have any idea of how to fix it please help.

Well,

this is the output from a debug run, and it shows that the 'dd .. | file -'
command is failing to detect it's a bzip2 file for some reason.

Maybe this should be assigned to the portage people as it's relying on the
rpm_unpack command to get things right.

+ src_unpack
+ local rpm_archive
+ for rpm_archive in '${A}'
+ rpm_unpack /var/tmp/portage/fcdsl-2.6.37/distdir/km_fcdsl-2.6-37.i586.rpm
+ local rpmfile rpmoff decompcmd
+ rpmfile=/var/tmp/portage/fcdsl-2.6.37/distdir/km_fcdsl-2.6-37.i586.rpm
+ '[' -z /var/tmp/portage/fcdsl-2.6.37/distdir/km_fcdsl-2.6-37.i586.rpm ']'
+ '[' -x /usr/bin/rpm2cpio -a -z '' ']'
++ rpmoffset
+ rpmoff=19719
+ '[' -z 19719 ']'
+ decompcmd='gzip -dc'
++ dd if=/var/tmp/portage/fcdsl-2.6.37/distdir/km_fcdsl-2.6-37.i586.rpm
skip=19719 bs=1 count=3
++ file -
++ grep bzip2
+ '[' -n '/dev/stdin: bzip2 compressed data' ']'
+ decompcmd='bzip2 -dc'
+ dd ibs=19719 skip=1
if=/var/tmp/portage/fcdsl-2.6.37/distdir/km_fcdsl-2.6-37.i586.rpm
+ bzip2 -dc
+ cpio -idmu --no-preserve-owner --quiet
+ return 0
+ for rpm_archive in '${A}'
+ rpm_unpack /var/tmp/portage/fcdsl-2.6.37/distdir/avm_fcdsl-2.6-37.i586.rpm
+ local rpmfile rpmoff decompcmd
+ rpmfile=/var/tmp/portage/fcdsl-2.6.37/distdir/avm_fcdsl-2.6-37.i586.rpm
+ '[' -z /var/tmp/portage/fcdsl-2.6.37/distdir/avm_fcdsl-2.6-37.i586.rpm ']'
+ '[' -x /usr/bin/rpm2cpio -a -z '' ']'
++ rpmoffset
+ rpmoff=7083
+ '[' -z 7083 ']'
+ decompcmd='gzip -dc'
++ dd if=/var/tmp/portage/fcdsl-2.6.37/distdir/avm_fcdsl-2.6-37.i586.rpm
skip=7083 bs=1 count=3
++ file -
++ grep bzip2
+ '[' -n '' ']'
+ dd ibs=7083 skip=1
if=/var/tmp/portage/fcdsl-2.6.37/distdir/avm_fcdsl-2.6-37.i586.rpm
+ gzip -dc
+ cpio -idmu --no-preserve-owner --quiet

gzip: stdin: not in gzip format
cpio: premature end of archive
+ return 1
+ diefunc src_unpack 71 1 'failed to unpack avm_fcdsl-2.6-37.i586.rpm file'
+ local funcname=src_unpack lineno=71 exitcode=1
+ shift 3
+ echo

+ echo '!!! ERROR: net-dialup/fcdsl-2.6.37 failed.'
!!! ERROR: net-dialup/fcdsl-2.6.37 failed.

liquid: have you seen this failure with rpm before and any idea how to fix it?

(In reply to comment #1)
> this bug is known to us. on repetitive runs it fails sometimes and sometimes
> not. If you have any idea of how to fix it please help.

I don't have a clean solution I'm afraid but it seems it's an issue with piping
dd's output directly to "file". The following test case which is near to what
rpm.eclass does sooner or later causes the error, too:

export verbose=true
for (( x=0 ; x<=1000; x++ )); do 
    n=`dd if=/usr/portage/distfiles/avm_fcdsl-2.6-37.i586.rpm skip=7083 bs=1
count=3 2>/dev/null | file -`;
    if [ -n "`echo ${n} | grep bzip2`" ]; then 
        if $verbose; then 
            echo -n "${x}:"; echo ${n}; 
        else 
            true; 
        fi; 
    else 
        echo -n "FAILURE at ${x}: "; echo ${n}; 
        break; 
    fi; 
done   

This is the error I get and it's always the same (not the same iteration, of
course):

FAILURE at 479: /dev/stdin: very short file (no magic)

If I pipe dd's output to a file, though, and pipe its contents to file, the
error never occurs (tested several times with 10000 iterations):

n=`dd if=/usr/portage/distfiles/avm_fcdsl-2.6-37.i586.rpm skip=7083 bs=1
count=3 2>/dev/null >dd_out; cat dd_out | file -`;

I'm currently doing repetitive "rm -rf /var/tmp/...; ebuild ... unpack"s with
the following modification in rpm.eclass:

if [ -n "`dd if=${rpmfile} skip=${rpmoff} bs=1 count=3 2>/dev/null >
${WORKDIR}/dd_out; cat ${WORKDIR}/dd_out | file - | grep bzip2`" ];

So far, 75 out of 1000 iterations done, the error didn't occur. I'm aware that
this is ugly but maybe it helps to track down the problem.

(In reply to comment #4)
> If I pipe dd's output to a file, though, and pipe its contents to file, the
> error never occurs (tested several times with 10000 iterations):
[...]
> So far, 75 out of 1000 iterations done, the error didn't occur. 

With the rpm.eclass modification I described, I've now successfully unpacked
1000 times. This definitely doesn't explain why this error occurs and it
doesn't prove it won't happen again even with my changed rpm.eclass but
empirically it seems to be reliable at least:

1000
 * km_fcdsl-2.6-37.i586.rpm MD5 ;-) ...                                        
                                [ ok ]
 * km_fcdsl-2.6-37.i586.rpm size ;-) ...                                       
                                [ ok ]
[...]
 * Converting 'drdsl.ini' to UTF-8
>>> Source unpacked.

Reassigned to rpm.eclass maintainer.

Oddly enough, I cannot reproduce this anymore.
Please try to reproduce it on a fully updated system.

Since genstef cannot reproduce it either, I close this as FIXED.
I think the last gzip update (app-arch/gzip-1.3.5-r9) fixed this problem.

I've just re-tried this today and it still happens here with a fresh portage
tree and a complete rebuild. So reopening.

It's easy to reproduce too. As in comment #4 just keep doing this...

dd if=avm_fcdsl-2.6-37.i586.rpm skip=7083 bs=1 count=3 | file -

and there'll be some failures. You'll either get this....

dd if=avm_fcdsl-2.6-37.i586.rpm skip=7083 bs=1 count=3 | file -
3+0 records in
3+0 records out
3 bytes (3 B) copied, 8.1e-05 s, 37.0 kB/s
/dev/stdin: bzip compressed data

Which is correct, or this....

dd if=avm_fcdsl-2.6-37.i586.rpm skip=7083 bs=1 count=3 | file -
/dev/stdin: very short file (no magic)
3+0 records in
3+0 records out
3 bytes (3 B) copied, 8.1e-05 s, 37.0 kB/s

which is busted.

Just for informational purposes, here's the versions of dd and file I've got...

dd --version
dd (coreutils) 6.4
Copyright (C) 2006 Free Software Foundation, Inc.
This is free software.  You may redistribute copies of it under the terms of
the GNU General Public License <http://www.gnu.org/licenses/gpl.html>.
There is NO WARRANTY, to the extent permitted by law.

Written by Paul Rubin, David MacKenzie, and Stuart Kemp.

# file --version
file-4.17
magic file from /usr/share/misc/file/magic

I cannot reproduce it with neither of the following commands:
1) dd if=/usr/portage/distfiles/avm_fcdsl-2.6-37.i586.rpm skip=7083 bs=1
count=3 | file -
2) ebuild fcdsl-2.6.37.ebuild clean unpack

I've executed both commands several times with the same result: total success.
I have coreutils-6.4.

Using my test case from comment #4 and setting it to 1000, 10000, 15000 and
finally 50000 iterations I haven't experienced a single failure either.

sys-apps/coreutils-6.7 (acl nls -selinux -static) which includes dd 6.7 and
file 4.18.

I'm going to start another test run on an up-to-date stable system in a few
minutes. 

(In reply to comment #12)
> I've executed both commands several times with the same result: total success.
> I have coreutils-6.4.

Using my test case from comment #4 I *can* definitely reproduce this on a
current stable system within 1000 iterations. 

Looks like <=file-4.17-r1 (not coreutils' dd) causes this problem because I can
reproduce the problem when downgrading (on an unstable system) from 4.18. After
upgrading again the problem is gone.

On a stable system upgrading to 4.18 but keeping coreutils-6.4 seems to fix the
problem, too. (The long-term tests are still running but no errors during the
first 1000 iterations.)

I predict some boredom on my part tomorrow which I might prevent by comparing
the sources. :-)

Ok, file's upstream, Christos Zoulas, has kindly confirmed that this problem is
caused by a bug in file 4.17:

[Answering my inquiry if he was aware of this problem]
"Yes, the problem was that sometimes file would not wait long enough
for input to appear and would exit too soon."

Time to stabilise 4.18?

base-system issue. They need to CC the arches for stabling

*** Bug 157946 has been marked as a duplicate of this bug. ***

x86 filed

Also is sparc.

ppc stable

ppc64 stable

Stable for HPPA.

AMD64 gone ;)

Stable on MIPS.

all set