Distfile nwresources129.tar.gz has incorrect sha256 digest entry in Manifest, though it has correct md5 and rmd160 digests. $ sha256sum /var/gentoo/distfiles/nwresources129.tar.gz 4da35c38c63d7cbe40047833103fe0d56f417d1895fb572d52c23a873619f537 /var/gentoo/distfiles/nwresources129.tar.gz $ grep -r nwresources * | grep SHA256 files/digest-nwn-data-1.29-r1:SHA256 40e7bb8aebae931c2ce05f199ed15714845b403c4a6d9fab88092e784214bee2 nwresources129.tar.gz 1212356127 files/digest-nwn-data-1.29:SHA256 40e7bb8aebae931c2ce05f199ed15714845b403c4a6d9fab88092e784214bee2 nwresources129.tar.gz 1212356127 Manifest:DIST nwresources129.tar.gz 1212356127 RMD160 1a986d12bc3153ff7646053e60d610155944b5ac SHA1 6f399dbf876f357a57096996447d4b9fa68bcb58 SHA256 40e7bb8aebae931c2ce05f199ed15714845b403c4a6d9fab88092e784214bee2 * Checking MD5 for nwresources129.tar.gz ... [ ok ] * Checking RMD160 for nwresources129.tar.gz ... [ ok ] * Checking SHA256 for nwresources129.tar.gz ... [ !! ]
This is strange. I am getting the same results. wolf31o2@inertia /usr/local/portage/games-rpg/nwn-data $ rm files/digest-nwn-data-1.29* wolf31o2@inertia /usr/local/portage/games-rpg/nwn-data $ rm Manifest wolf31o2@inertia /usr/local/portage/games-rpg/nwn-data $ ebuild nwn-data-1.29.ebuild digest >>> Creating Manifest for /usr/local/portage/games-rpg/nwn-data wolf31o2@inertia /usr/local/portage/games-rpg/nwn-data $ ll total 44 -rw-r--r-- 1 wolf31o2 users 4393 2007-01-11 12:06 ChangeLog drwxr-xr-x 2 wolf31o2 users 4096 2007-01-31 17:07 files -rw-r--r-- 1 wolf31o2 users 3567 2007-01-31 17:07 Manifest -rw-r--r-- 1 wolf31o2 users 254 2007-01-11 12:06 metadata.xml -rw-r--r-- 1 wolf31o2 users 6787 2007-01-11 12:06 nwn-data-1.29.ebuild -rw-r--r-- 1 wolf31o2 users 15925 2007-01-11 14:17 nwn-data-1.29-r1.ebuild wolf31o2@inertia /usr/local/portage/games-rpg/nwn-data $ cat Manifest | grep nwresources DIST nwresources129.tar.gz 1212356127 RMD160 1a986d12bc3153ff7646053e60d610155944b5ac SHA1 6f399dbf876f357a57096996447d4b9fa68bcb58 SHA256 40e7bb8aebae931c2ce05f199ed15714845b403c4a6d9fab88092e784214bee2 wolf31o2@inertia /usr/local/portage/games-rpg/nwn-data $ grep nwresources files/digest-nwn-data-1.29-r1 | grep SHA256 SHA256 40e7bb8aebae931c2ce05f199ed15714845b403c4a6d9fab88092e784214bee2 nwresources129.tar.gz 1212356127 wolf31o2@inertia /usr/local/portage/games-rpg/nwn-data $ sha256sum /usr/portage/distfiles/nwresources129.tar.gz 4da35c38c63d7cbe40047833103fe0d56f417d1895fb572d52c23a873619f537 /usr/portage/distfiles/nwresources129.tar.gz wolf31o2@inertia /usr/local/portage/games-rpg/nwn-data $ shash -a SHA256 /usr/portage/distfiles/nwresources129.tar.gz # SHA256 HASH 4da35c38c63d7cbe40047833103fe0d56f417d1895fb572d52c23a873619f537 /usr/portage/distfiles/nwresources129.tar.gz Portage: How are the SHA256 digests calculated?
(In reply to comment #1) > Portage: How are the SHA256 digests calculated? It uses pycrypto. We've had problems with pycrypto and sha256 before (bug 131293).
OK. I am definitely able to reproduce this with games-rpg/nwn-data and: [ebuild R ] sys-apps/portage-2.1.2-r7 [ebuild R ] dev-python/pycrypto-2.0.1-r5
@wolflo, kindly state the arch you're running, gcc, etc. If you know the arch/gcc of the system that generated the initial sha256, would be useful; it's paranoid, but if pycrypto is again screwing up, that info will help nail down the issue
$ emerge --info Portage 2.1.2-r9 (default-linux/x86/2006.1, gcc-4.1.2, glibc-2.5-r0, 2.6.18-gentoo-r3 i686) ================================================================= System uname: 2.6.18-gentoo-r3 i686 Intel(R) Core(TM)2 CPU T7200 @ 2.00GHz Gentoo Base System release 1.12.9 Timestamp of tree: Mon, 19 Feb 2007 15:30:01 +0000 distcc 2.18.3 i686-pc-linux-gnu (protocols 1 and 2) (default port 3632) [enabled] dev-java/java-config: 1.3.7, 2.0.31-r3 dev-lang/python: 2.4.4 dev-python/pycrypto: 2.0.1-r5 sys-apps/sandbox: 1.2.18.1 sys-devel/autoconf: 2.13, 2.61 sys-devel/automake: 1.4_p6, 1.5, 1.6.3, 1.7.9-r1, 1.8.5-r3, 1.9.6-r2, 1.10 sys-devel/binutils: 2.17 sys-devel/gcc-config: 1.3.14 sys-devel/libtool: 1.5.22 virtual/os-headers: 2.6.20 ACCEPT_KEYWORDS="x86 ~x86" AUTOCLEAN="yes" CBUILD="i686-pc-linux-gnu" CFLAGS="-O2 -march=prescott -msse3 -pipe -fomit-frame-pointer" CHOST="i686-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/share/X11/xkb" CONFIG_PROTECT_MASK="/etc/env.d /etc/env.d/java/ /etc/gconf /etc/java-config/vms/ /etc/revdep-rebuild /etc/splash /etc/terminfo" CXXFLAGS="-O2 -march=prescott -msse3 -pipe -fomit-frame-pointer" DISTDIR="/usr/portage/distfiles" FEATURES="ccache collision-protect cvs distcc distlocks metadata-transfer sandbox sfperms sign" GENTOO_MIRRORS="ftp://ftp.ussg.iu.edu/pub/linux/gentoo ftp://gentoo.mirrors.tds.net/gentoo http://ftp.snt.utwente.nl/pub/os/linux/gentoo http://ftp.heanet.ie/pub/gentoo/ http://85.25.128.62" LC_ALL="en_US.UTF-8" LINGUAS="en" MAKEOPTS="-j8" PKGDIR="/usr/portage/packages" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --delete-after --stats --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages" PORTAGE_TMPDIR="/var/tmp" PORTDIR="/usr/portage" PORTDIR_OVERLAY="/usr/local/portage /var/svnroot/wolf31o2 /usr/portage/local/layman/vmware /usr/portage/local/layman/efika /usr/portage/local/layman/nouveau" SYNC="rsync://rsync.gentoo.org/gentoo-portage" USE="X acpi alsa branding cairo cdr crypt cups dbus dlloader dvd dvdr eds encode esd ethereal evo fam firefox gif gnome gphoto2 gstreamer gtk gtk2 hal java jpeg ldap mmx mozbranding mp3 mpeg ncurses nls nocd nptl nptlonly ogg openal opengl oss pam pcmcia pdf perl pic png pnp ppds quicktime readline samba snmp spell sse sse2 ssl svg tcpd tiff truetype trusted udev unicode usb videos vorbis win32codecs wireshark x86 xml xorg xv xvid zlib" ALSA_CARDS="hda-intel" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mulaw multi null plug rate route share shm softvol" CAMERAS="kodak" ELIBC="glibc" INPUT_DEVICES="evdev keyboard mouse synaptics" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LINGUAS="en" USERLAND="GNU" VIDEO_CARDS="nvidia nv vesa" Unset: CTARGET, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LANG, LDFLAGS, PORTAGE_RSYNC_EXTRA_OPTS I recently switched the CPU from a Core Duo to a Core 2 Duo, so everything is still setup for Core. I also recently upgraded my world, so I've rebuilt pycrypto and am re-running the digest. I made sure to clear out the pyc/pyo files under /usr/lib/python2.4/site-packages/Crypto, too, just in case. wolf31o2@inertia ~ $ shash -a SHA256 /usr/portage/distfiles/nwresources129.tar.gz # SHA256 HASH 4da35c38c63d7cbe40047833103fe0d56f417d1895fb572d52c23a873619f537 /usr/portage/distfiles/nwresources129.tar.gz wolf31o2@inertia ~ $ sha256sum /usr/portage/distfiles/nwresources129.tar.gz 4da35c38c63d7cbe40047833103fe0d56f417d1895fb572d52c23a873619f537 /usr/portage/distfiles/nwresources129.tar.gz wolf31o2@inertia /var/cvsroot/gentoo-x86/games-rpg/nwn-data $ ebuild nwn-data-1.29-r1.ebuild digest Appending /var/cvsroot/gentoo-x86 to PORTDIR_OVERLAY... >>> Creating Manifest for /var/cvsroot/gentoo-x86/games-rpg/nwn-data wolf31o2@inertia /var/cvsroot/gentoo-x86/games-rpg/nwn-data $ grep SHA256 Manifest | grep nwresources DIST nwresources129.tar.gz 1212356127 RMD160 1a986d12bc3153ff7646053e60d610155944b5ac SHA1 6f399dbf876f357a57096996447d4b9fa68bcb58 SHA256 40e7bb8aebae931c2ce05f199ed15714845b403c4a6d9fab88092e784214bee2
What's the output of python -c 'from Crypto.Hash import SHA256; print SHA256.new(open("nwresources129.tar.gz", "r").read()).hexdigest()'
wolf31o2@inertia ~ $ python -c 'from Crypto.Hash import SHA256; print SHA256.new(open("/usr/portage/distfiles/nwresources129.tar.gz", "r").read()).hexdigest()' 40e7bb8aebae931c2ce05f199ed15714845b403c4a6d9fab88092e784214bee2
*** Bug 175585 has been marked as a duplicate of this bug. ***
Also seeing this for the 700Mb ophcrack tables (available on distfiles.gentoo.org as SSTIC04-5k.zip). SHA256 from pycrypto is wrong and disagrees with sha256sum from coreutils and also the hashlib sha256 implementation. The original hash (the one in Manifest) was made using pycrypto awhile ago and was also wrong. The error now occurs presumably because I've got python-2.5 and so will have started using the hashlib library instead as of 2.1.3_rc6... I can attach my emerge --info if it'll help?
Created attachment 125518 [details, diff] fix from upstream This patch from upstream seems to correct the problem: http://pycrypto.cvs.sourceforge.net/pycrypto/crypto/src/SHA256.c?r1=1.4&r2=1.5 Without the patch, the sha256 digest for SSTIC04-5k.zip is: 910cfab447d273564a374d62620d50812d0623657905835da152d53c3542552f With the patch, the sha256 digest for SSTIC04-5k.zip is: 9ba2c9e6faeb658a77a342a2d45c47e593daae9e087be3254eb1b050b324cf10 The patch brings the sha256 digest into agreement with that calculated by sha256sum from coreutils. @python herd: can we do a pycrypto revbump with this patch?
pycrypto-2.0.1-r6 is in cvs with this patch included.
I guess we can remove ourselves from this. I've manually updated the nwresources129 digest/Manifest a long time ago.
Similarly, I've also re-digested SSTIC04-5k.zip. Perhaps this bug can be marked as resolved?
The new version is stable on all the really relevant archs, so I think we can consider this fixed.
*** Bug 192603 has been marked as a duplicate of this bug. ***
