Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 69462 - portage sometimes cant handle building digests
Summary: portage sometimes cant handle building digests
Status: RESOLVED FIXED
Alias: None
Product: Portage Development
Classification: Unclassified
Component: Core - Ebuild Support (show other bugs)
Hardware: All All
: High major
Assignee: Portage team
URL:
Whiteboard:
Keywords:
: 69922 71078 78431 (view as bug list)
Depends on:
Blocks:
 
Reported: 2004-10-29 15:00 UTC by SpanKY
Modified: 2005-02-28 04:48 UTC (History)
3 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description SpanKY gentoo-dev 2004-10-29 15:00:13 UTC
noticed this while playing with gcc ...

root@vapier 0 gcc # cd /usr/portage/sys-devel/gcc
root@vapier 0 gcc # export USE=-boundschecking
root@vapier 0 gcc # rm files/digest-gcc-*
root@vapier 0 gcc # ebuild gcc-3.4.2-r3.ebuild digest
>>> Generating digest file...
<<< gcc-3.4.2.tar.bz2
<<< gcc-3.4.2-branch-update-20041025.patch.bz2
<<< protector-3.4.1-1.tar.gz
<<< gcc-3.4.2-manpages.tar.bz2
<<< gcc-3.4.2-patches-1.2.tar.bz2
<<< gcc-3.4.0-piepatches-v8.7.6.5.tar.bz2
<<< bounds-checking-gcc-3.4.2-1.00.patch.bz2
!!! We have a source URI, but no file...
!!! File: /usr/portage/distfiles/bounds-checking-gcc-3.4.2-1.00.patch.bz2
root@vapier 0 gcc # ebuild gcc-3.4.2-r3.ebuild fetch 
!!! No package digest file found: /usr/portage/sys-devel/gcc/files/digest-gcc-3.4.2-r3
!!! Type "ebuild foo.ebuild digest" to generate it.
root@vapier 0 gcc # export USE=boundschecking
root@vapier 0 gcc # ebuild gcc-3.4.2-r3.ebuild fetch 
< bounds-checking-gcc-3.4.2-1.00.patch.bz2 is downloaded properly >
!!! No package digest file found: /usr/portage/sys-devel/gcc/files/digest-gcc-3.4.2-r3
!!! Type "ebuild foo.ebuild digest" to generate it.
root@vapier 0 gcc # ebuild gcc-3.4.2-r3.ebuild digest
>>> Generating digest file...
<<< gcc-3.4.2.tar.bz2
<<< gcc-3.4.2-branch-update-20041025.patch.bz2
<<< protector-3.4.1-1.tar.gz
<<< gcc-3.4.2-manpages.tar.bz2
<<< gcc-3.4.2-patches-1.2.tar.bz2
<<< gcc-3.4.0-piepatches-v8.7.6.5.tar.bz2
<<< bounds-checking-gcc-3.4.2-1.00.patch.bz2
>>> Generating manifest file...
< snip >
root@vapier 0 gcc # export USE=-boundschecking
root@vapier 0 gcc # rm /usr/portage/distfiles/bounds-checking-gcc-3.4.2-1.00.patch.bz2 
root@vapier 0 gcc # ebuild gcc-3.4.2-r3.ebuild digest
>>> Generating digest file...
<<< gcc-3.4.2.tar.bz2
<<< gcc-3.4.2-branch-update-20041025.patch.bz2
<<< protector-3.4.1-1.tar.gz
<<< gcc-3.4.2-manpages.tar.bz2
<<< gcc-3.4.2-patches-1.2.tar.bz2
<<< gcc-3.4.0-piepatches-v8.7.6.5.tar.bz2
<<< bounds-checking-gcc-3.4.2-1.00.patch.bz2
Traceback (most recent call last):
  File "/usr/sbin/ebuild", line 49, in ?
    a=portage.doebuild(pargs[0],x,getroot(),tmpsettings,debug=debug,cleanup=cleanup)
  File "/usr/lib/portage/pym/portage.py", line 2520, in doebuild
    return (not digestgen(aalist,mysettings,overwrite=1))
  File "/usr/lib/portage/pym/portage.py", line 1950, in digestgen
    mydigests=digestCreate(myarchives, basedir, oldDigest=myolddigest)
  File "/usr/lib/portage/pym/portage.py", line 1886, in digestCreate
    raise portage_exception.DigestException, "Size mismatch during checksums"
portage_exception.DigestException: 'Size mismatch during checksums'


Portage 2.0.51-r2 (default-linux/x86/2004.2, gcc-3.4.2, glibc-2.3.4.20041006-r0, 2.6.9 i686)
=================================================================
System uname: 2.6.9 i686 Intel(R) Pentium(R) 4 CPU 1.80GHz
Gentoo Base System version 1.6.4
distcc 2.18 i686-pc-linux-gnu (protocols 1 and 2) (default port 3632) [disabled]
ccache version 2.3 [enabled]
Autoconf: sys-devel/autoconf-2.59-r6,sys-devel/autoconf-2.13
Automake: sys-devel/automake-1.5,sys-devel/automake-1.8.5-r2,sys-devel/automake-1.7.9,sys-devel/automake-1.6.3,sys-devel/automake-1.4_p6
Binutils: sys-devel/binutils-1,sys-devel/binutils-1-r1,sys-devel/binutils-2.15.92.0.2-r1,sys-devel/binutils-1-r2,sys-devel/binutils-1-r3,sys-devel/binutils-1-r4,sys-devel/binutils-1-r5,sys-devel/binutils-2.15.92.0.2-r2
Headers:  sys-kernel/linux26-headers-1,sys-kernel/linux26-headers-2.6.8.1-r1
Libtools: sys-devel/libtool-1.5.2-r5
ACCEPT_KEYWORDS="x86 ~x86"
AUTOCLEAN="yes"
CFLAGS="-pipe -march=pentium4 -O2 -frename-registers -fomit-frame-pointer -mmmx -msse -msse2 -fdelete-null-pointer-checks -funroll-loops -ffast-math -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE"
CHOST="i686-pc-linux-gnu"
COMPILER=""
CONFIG_PROTECT="/etc /opt/glftpd/etc /usr/X11R6/lib/X11/xkb /usr/kde/2/share/config /usr/kde/3.3/env /usr/kde/3.3/share/config /usr/kde/3.3/shutdown /usr/kde/3/share/config /usr/lib/mozilla/defaults/pref /usr/share/config /usr/share/texmf/dvipdfm/config/ /usr/share/texmf/dvips/config/ /usr/share/texmf/tex/generic/config/ /usr/share/texmf/tex/platex/config/ /usr/share/texmf/xdvi/ /var/qmail/control"
CONFIG_PROTECT_MASK="/etc/gconf /etc/terminfo /etc/env.d"
CXXFLAGS="-pipe -march=pentium4 -O2 -frename-registers -fomit-frame-pointer -mmmx -msse -msse2 -fdelete-null-pointer-checks -funroll-loops -ffast-math -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE"
DISTDIR="/usr/portage/distfiles"
FEATURES="ccache distlocks noauto noinfo sandbox sfperms sign"
GENTOO_MIRRORS="ftp://ftp.ussg.iu.edu/pub/linux/gentoo"
MAKEOPTS="-j4"
PKGDIR="/mnt/space/packages"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
PORTDIR_OVERLAY="/usr/local/portage"
SYNC="rsync://gentoo/gentoo-portage"
USE="X aalib acl acpi adns alsa apache2 avi bitmap-fonts cdr crypt cscope cups dedicated dga directfb dvd dvdr emu10k1 encode esd f77 fam fbcon flac flash foomaticdb gd gd-external gdbm gif gpm gstreamer gtk gtk2 imagemagick imap imlib ipv6 joystick jpeg kde libwww mad maildir mikmod mmx moznocompose moznoirc moznomail mpeg mysql nas ncurses nls nocardbus nptl nptlonly offensive oggvorbis opengl oss pam pdflib perl php pic plotutils png pnp python quicktime readline samba sdl slp snmp socks5 speex spell sqlite sse ssl svga tcltk tcpd tiff truetype wavelan x86 xine xml xml2 xmms xv zlib video_cards_nvidia"
Comment 1 Jason Stubbs (RETIRED) gentoo-dev 2004-10-29 19:19:01 UTC
I can't reproduce this...

jason@localhost ~/gentoo-x86/sys-devel/gcc $ export USE="-boundschecking"
jason@localhost ~/gentoo-x86/sys-devel/gcc $ ls gcc-3.4.2-r3.ebuild files/digest-gcc-3.4.2-r3
ls: files/digest-gcc-3.4.2-r3: No such file or directory
gcc-3.4.2-r3.ebuild
jason@localhost ~/gentoo-x86/sys-devel/gcc $ ebuild gcc-3.4.2-r3.ebuild digest
>>> Downloading http://gentoo.osuosl.org/distfiles/bounds-checking-gcc-3.4.2-1.00.patch.bz2
--11:20:34--  http://gentoo.osuosl.org/distfiles/bounds-checking-gcc-3.4.2-1.00.patch.bz2
Comment 2 SpanKY gentoo-dev 2004-10-30 13:22:29 UTC
*shrug*

i was able to reproduce it one three different boxes ... did you remove *just* that digest, or did you remove *all* of them ?
Comment 3 Jason Stubbs (RETIRED) gentoo-dev 2004-10-30 19:55:18 UTC
jason@localhost ~/gentoo-x86/sys-devel/gcc $ export PORTDIR="/home/jason/gentoo-x86/"
jason@localhost ~/gentoo-x86/sys-devel/gcc $ export FEATURES="-* ccache distlocks noauto noinfo sandbox sfperms sign"
jason@localhost ~/gentoo-x86/sys-devel/gcc $ export USE="-boundschecking"
jason@localhost ~/gentoo-x86/sys-devel/gcc $ rm files/digest-gcc-*
jason@localhost ~/gentoo-x86/sys-devel/gcc $ mv /home/distfiles/bounds-checking-gcc-3.4.2-1.00.patch.bz2 /home/jason
jason@localhost ~/gentoo-x86/sys-devel/gcc $ ebuild gcc-3.4.2-r3.ebuild digest
>>> Downloading http://gentoo.osuosl.org/distfiles/bounds-checking-gcc-3.4.2-1.00.patch.bz2
< snip >
Exiting due to signal
jason@localhost ~/gentoo-x86/sys-devel/gcc $ mv /home/jason/bounds-checking-gcc-3.4.2-1.00.patch.bz2 /home/distfiles/
jason@localhost ~/gentoo-x86/sys-devel/gcc $ ebuild gcc-3.4.2-r3.ebuild digest
>>> Generating digest file...
<<< gcc-3.4.2.tar.bz2
<<< gcc-3.4.2-branch-update-20041025.patch.bz2
<<< protector-3.4.1-1.tar.gz
<<< gcc-3.4.2-manpages.tar.bz2
<<< gcc-3.4.2-patches-1.2.tar.bz2
<<< gcc-3.4.0-piepatches-v8.7.6.5.tar.bz2
<<< bounds-checking-gcc-3.4.2-1.00.patch.bz2
>>> Generating manifest file...
< snip >


I was able to reproduce the "We have a source URI, but no file..." only by deleting bounds-checking-gcc-3.4.2-1.00.patch.bz2 after "ebuild digest" had downloaded it and was downloading other files.

I'm unable to reproduce the traceback at all and, tracing back through the code, can't even see how it's theoretically possible.

Nick, can you explain this at all?
Comment 4 Brian Harring (RETIRED) gentoo-dev 2004-11-03 01:29:59 UTC
*** Bug 69922 has been marked as a duplicate of this bug. ***
Comment 5 Henrik Sankala 2004-11-05 08:07:45 UTC
I can reproduce this error with the dnetc ebuilds:
>>> Downloading http://http.distributed.net/pub/dcti/v2.9008/dnetc490-linux-x86-elf-uclibc.tar.gz
...
>>> Generating digest file...
<<< dnetc490-linux-x86-elf-uclibc.tar.gz
<<< dnetc490-linux-sparc.tar.gz
!!! We have a source URI, but no file...
!!! File: /usr/portage/distfiles/dnetc490-linux-sparc.tar.gz

The "ebuild ... digest" command only downloads the sources for our current arch, but then tries to create digests for all archs.
Comment 6 SpanKY gentoo-dev 2004-11-15 14:43:10 UTC
*** Bug 71078 has been marked as a duplicate of this bug. ***
Comment 7 ta2002 2004-11-18 09:39:01 UTC
I think this reflects a recent change in portage.

I made a modification to xorg-x11, and put it in
/usr/local/portage.

A month ago "ebuild xorg-x11-6.8.0-r1.ebuild digest"
worked fine, but now, I get the "!!! We have a source
URI, but no file..." message listing the source file
only needed for the "doc" USE flag (which I do not
have set).
Comment 8 Daniel Ahlberg (RETIRED) gentoo-dev 2005-01-01 05:56:49 UTC
It seems the error only occurs if SRC_URI contains more than one file and that file depends on whetever a use flag is set. If I download all the files in SRC_URI manually I can create the digest. (Using 2.0.51-r8)
Comment 9 SpanKY gentoo-dev 2005-01-17 15:15:40 UTC
*** Bug 78431 has been marked as a duplicate of this bug. ***
Comment 10 Nicholas Jones (RETIRED) gentoo-dev 2005-01-17 15:19:43 UTC
Try these as solutions:

emerge --fetch-all-uri
or
copy the old digest.
Comment 11 Henning Rohde 2005-01-17 16:42:13 UTC
sorry, "emerge --fetch-all-uri" does not work in case of bug #78431 because emerge needs the digest as a prerequisite.

It's a chicken'n'egg problem: I can't generate the digest because some URIs are missing, and I can't download the missing URIs because the manifest aka digest is missing...

(should have mentioned that I do alter $PR after modifiing the ebuild)
Comment 12 Brian Harring (RETIRED) gentoo-dev 2005-02-28 01:28:37 UTC
Jason, did you get in the reversion for portage that knocks it back to generating complete digests, rather then incomplete?
That's what this bug is...
Comment 13 Jason Stubbs (RETIRED) gentoo-dev 2005-02-28 04:48:08 UTC
Yeah, should be back to 100% on this since ~r13. Existing digest entries are reused and missing ones are automatically downloaded/generated.