144624 2006-08-21 04:24 0000 stabilize sys-apps/file-4.18 (WAS: net-dialup/fcdsl-2.6.37 fails to unpack rpm) 2007-02-07 04:38:39 0000 1 1 1 Unclassified Gentoo Linux Applications 2006.0 All Linux RESOLVED FIXED P2 normal --- 157946 1 alanh@fairlite.co.uk alpha@gentoo.org philantrop@gentoo.org alanh@fairlite.co.uk 2006-08-21 04:24:34 0000 emerging fcdsl fails, as it appears to be trying to unpack a gzip archive rather than the downloaded rpm file. emerge fcdsl Calculating dependencies... done! >>> Emerging (1 of 1) net-dialup/fcdsl-2.6.37 to / >>> checking ebuild checksums ;-) >>> checking auxfile checksums ;-) >>> checking miscfile checksums ;-) >>> checking km_fcdsl-2.6-37.i586.rpm ;-) >>> checking avm_fcdsl-2.6-37.i586.rpm ;-) * Determining the location of the kernel source code * Found kernel source directory: * /usr/src/linux * Found sources for kernel version: * 2.6.16-hardened-r11 * Checking for suitable kernel configuration options... [ ok ] * * You can control the modules which are built with the variable * FCDSL_CARDS which should contain a blank separated list * of a selection from the following cards: * fcdsl2 fcdslsl fcdslusb fcdslslusb fcdslusb2 fcdsl fcdslusba >>> Unpacking source... gzip: stdin: not in gzip format cpio: premature end of archive !!! ERROR: net-dialup/fcdsl-2.6.37 failed. Call stack: ebuild.sh, line 1539: Called dyn_unpack ebuild.sh, line 711: Called src_unpack fcdsl-2.6.37.ebuild, line 71: Called die !!! failed to unpack avm_fcdsl-2.6-37.i586.rpm file !!! If you need support, post the topmost build error, and the call stack if relevant. genstef@gentoo.org 2006-08-21 05:10:55 0000 this bug is known to us. on repetitive runs it fails sometimes and sometimes not. If you have any idea of how to fix it please help. alanh@fairlite.co.uk 2006-08-21 06:29:26 0000 Well, this is the output from a debug run, and it shows that the 'dd .. | file -' command is failing to detect it's a bzip2 file for some reason. Maybe this should be assigned to the portage people as it's relying on the rpm_unpack command to get things right. + src_unpack + local rpm_archive + for rpm_archive in '${A}' + rpm_unpack /var/tmp/portage/fcdsl-2.6.37/distdir/km_fcdsl-2.6-37.i586.rpm + local rpmfile rpmoff decompcmd + rpmfile=/var/tmp/portage/fcdsl-2.6.37/distdir/km_fcdsl-2.6-37.i586.rpm + '[' -z /var/tmp/portage/fcdsl-2.6.37/distdir/km_fcdsl-2.6-37.i586.rpm ']' + '[' -x /usr/bin/rpm2cpio -a -z '' ']' ++ rpmoffset + rpmoff=19719 + '[' -z 19719 ']' + decompcmd='gzip -dc' ++ dd if=/var/tmp/portage/fcdsl-2.6.37/distdir/km_fcdsl-2.6-37.i586.rpm skip=19719 bs=1 count=3 ++ file - ++ grep bzip2 + '[' -n '/dev/stdin: bzip2 compressed data' ']' + decompcmd='bzip2 -dc' + dd ibs=19719 skip=1 if=/var/tmp/portage/fcdsl-2.6.37/distdir/km_fcdsl-2.6-37.i586.rpm + bzip2 -dc + cpio -idmu --no-preserve-owner --quiet + return 0 + for rpm_archive in '${A}' + rpm_unpack /var/tmp/portage/fcdsl-2.6.37/distdir/avm_fcdsl-2.6-37.i586.rpm + local rpmfile rpmoff decompcmd + rpmfile=/var/tmp/portage/fcdsl-2.6.37/distdir/avm_fcdsl-2.6-37.i586.rpm + '[' -z /var/tmp/portage/fcdsl-2.6.37/distdir/avm_fcdsl-2.6-37.i586.rpm ']' + '[' -x /usr/bin/rpm2cpio -a -z '' ']' ++ rpmoffset + rpmoff=7083 + '[' -z 7083 ']' + decompcmd='gzip -dc' ++ dd if=/var/tmp/portage/fcdsl-2.6.37/distdir/avm_fcdsl-2.6-37.i586.rpm skip=7083 bs=1 count=3 ++ file - ++ grep bzip2 + '[' -n '' ']' + dd ibs=7083 skip=1 if=/var/tmp/portage/fcdsl-2.6.37/distdir/avm_fcdsl-2.6-37.i586.rpm + gzip -dc + cpio -idmu --no-preserve-owner --quiet gzip: stdin: not in gzip format cpio: premature end of archive + return 1 + diefunc src_unpack 71 1 'failed to unpack avm_fcdsl-2.6-37.i586.rpm file' + local funcname=src_unpack lineno=71 exitcode=1 + shift 3 + echo + echo '!!! ERROR: net-dialup/fcdsl-2.6.37 failed.' !!! ERROR: net-dialup/fcdsl-2.6.37 failed. genstef@gentoo.org 2006-08-21 11:02:30 0000 liquid: have you seen this failure with rpm before and any idea how to fix it? philantrop@gentoo.org 2006-08-22 01:02:29 0000 (In reply to comment #1) > this bug is known to us. on repetitive runs it fails sometimes and sometimes > not. If you have any idea of how to fix it please help. I don't have a clean solution I'm afraid but it seems it's an issue with piping dd's output directly to "file". The following test case which is near to what rpm.eclass does sooner or later causes the error, too: export verbose=true for (( x=0 ; x<=1000; x++ )); do n=`dd if=/usr/portage/distfiles/avm_fcdsl-2.6-37.i586.rpm skip=7083 bs=1 count=3 2>/dev/null | file -`; if [ -n "`echo ${n} | grep bzip2`" ]; then if $verbose; then echo -n "${x}:"; echo ${n}; else true; fi; else echo -n "FAILURE at ${x}: "; echo ${n}; break; fi; done This is the error I get and it's always the same (not the same iteration, of course): FAILURE at 479: /dev/stdin: very short file (no magic) If I pipe dd's output to a file, though, and pipe its contents to file, the error never occurs (tested several times with 10000 iterations): n=`dd if=/usr/portage/distfiles/avm_fcdsl-2.6-37.i586.rpm skip=7083 bs=1 count=3 2>/dev/null >dd_out; cat dd_out | file -`; I'm currently doing repetitive "rm -rf /var/tmp/...; ebuild ... unpack"s with the following modification in rpm.eclass: if [ -n "`dd if=${rpmfile} skip=${rpmoff} bs=1 count=3 2>/dev/null > ${WORKDIR}/dd_out; cat ${WORKDIR}/dd_out | file - | grep bzip2`" ]; So far, 75 out of 1000 iterations done, the error didn't occur. I'm aware that this is ugly but maybe it helps to track down the problem. philantrop@gentoo.org 2006-08-22 04:58:46 0000 (In reply to comment #4) > If I pipe dd's output to a file, though, and pipe its contents to file, the > error never occurs (tested several times with 10000 iterations): [...] > So far, 75 out of 1000 iterations done, the error didn't occur. With the rpm.eclass modification I described, I've now successfully unpacked 1000 times. This definitely doesn't explain why this error occurs and it doesn't prove it won't happen again even with my changed rpm.eclass but empirically it seems to be reliable at least: 1000 * km_fcdsl-2.6-37.i586.rpm MD5 ;-) ... [ ok ] * km_fcdsl-2.6-37.i586.rpm size ;-) ... [ ok ] [...] * Converting 'drdsl.ini' to UTF-8 >>> Source unpacked. mrness@gentoo.org 2006-09-09 09:07:15 0000 Reassigned to rpm.eclass maintainer. mrness@gentoo.org 2006-10-14 00:49:54 0000 Oddly enough, I cannot reproduce this anymore. Please try to reproduce it on a fully updated system. mrness@gentoo.org 2006-10-16 11:01:56 0000 Since genstef cannot reproduce it either, I close this as FIXED. I think the last gzip update (app-arch/gzip-1.3.5-r9) fixed this problem. alanh@fairlite.co.uk 2006-12-07 09:13:00 0000 I've just re-tried this today and it still happens here with a fresh portage tree and a complete rebuild. So reopening. alanh@fairlite.co.uk 2006-12-07 09:27:45 0000 It's easy to reproduce too. As in comment #4 just keep doing this... dd if=avm_fcdsl-2.6-37.i586.rpm skip=7083 bs=1 count=3 | file - and there'll be some failures. You'll either get this.... dd if=avm_fcdsl-2.6-37.i586.rpm skip=7083 bs=1 count=3 | file - 3+0 records in 3+0 records out 3 bytes (3 B) copied, 8.1e-05 s, 37.0 kB/s /dev/stdin: bzip compressed data Which is correct, or this.... dd if=avm_fcdsl-2.6-37.i586.rpm skip=7083 bs=1 count=3 | file - /dev/stdin: very short file (no magic) 3+0 records in 3+0 records out 3 bytes (3 B) copied, 8.1e-05 s, 37.0 kB/s which is busted. alanh@fairlite.co.uk 2006-12-07 09:31:25 0000 Just for informational purposes, here's the versions of dd and file I've got... dd --version dd (coreutils) 6.4 Copyright (C) 2006 Free Software Foundation, Inc. This is free software. You may redistribute copies of it under the terms of the GNU General Public License <http://www.gnu.org/licenses/gpl.html>. There is NO WARRANTY, to the extent permitted by law. Written by Paul Rubin, David MacKenzie, and Stuart Kemp. # file --version file-4.17 magic file from /usr/share/misc/file/magic mrness@gentoo.org 2006-12-11 09:37:40 0000 I cannot reproduce it with neither of the following commands: 1) dd if=/usr/portage/distfiles/avm_fcdsl-2.6-37.i586.rpm skip=7083 bs=1 count=3 | file - 2) ebuild fcdsl-2.6.37.ebuild clean unpack I've executed both commands several times with the same result: total success. I have coreutils-6.4. philantrop@gentoo.org 2006-12-11 09:53:18 0000 Using my test case from comment #4 and setting it to 1000, 10000, 15000 and finally 50000 iterations I haven't experienced a single failure either. sys-apps/coreutils-6.7 (acl nls -selinux -static) which includes dd 6.7 and file 4.18. I'm going to start another test run on an up-to-date stable system in a few minutes. philantrop@gentoo.org 2006-12-11 14:22:39 0000 (In reply to comment #12) > I've executed both commands several times with the same result: total success. > I have coreutils-6.4. Using my test case from comment #4 I *can* definitely reproduce this on a current stable system within 1000 iterations. Looks like <=file-4.17-r1 (not coreutils' dd) causes this problem because I can reproduce the problem when downgrading (on an unstable system) from 4.18. After upgrading again the problem is gone. On a stable system upgrading to 4.18 but keeping coreutils-6.4 seems to fix the problem, too. (The long-term tests are still running but no errors during the first 1000 iterations.) I predict some boredom on my part tomorrow which I might prevent by comparing the sources. :-) philantrop@gentoo.org 2006-12-12 11:10:09 0000 Ok, file's upstream, Christos Zoulas, has kindly confirmed that this problem is caused by a bug in file 4.17: [Answering my inquiry if he was aware of this problem] "Yes, the problem was that sometimes file would not wait long enough for input to appear and would exit too soon." Time to stabilise 4.18? genstef@gentoo.org 2006-12-12 12:53:08 0000 base-system issue. They need to CC the arches for stabling vapier@gentoo.org 2006-12-12 19:49:24 0000 *** Bug 157946 has been marked as a duplicate of this bug. *** fauli@gentoo.org 2006-12-12 23:16:31 0000 x86 filed fmccor@gentoo.org 2006-12-13 04:20:49 0000 Also is sparc. dertobi123@gentoo.org 2006-12-13 08:07:30 0000 ppc stable ranger@gentoo.org 2006-12-13 17:53:29 0000 ppc64 stable jer@gentoo.org 2006-12-13 18:19:51 0000 Stable for HPPA. welp@gentoo.org 2006-12-14 13:08:06 0000 AMD64 gone ;) ahf@0x90.dk 2006-12-22 12:11:26 0000 Stable on MIPS. vapier@gentoo.org 2007-02-07 04:38:39 0000 all set