Bug 20544 – Checking MD5 sums immediately on fetching for unstable net connections

• Status: RESOLVED FIXED
• Product: Portage Development

[Description Uriah Heep 2003-05-06 14:44:54 UTC]

As explained in the thread on forums.gentoo.org:

http://forums.gentoo.org/viewtopic.php?p=311759#311759

The following patch will check MD5 sums on freshly downloaded packages/verify MD5 sums on previously downloaded packages so that users who have an intermittant net connection can do things like "emerge -fu kde" (which sometimes pulls as many as 90+ packages) and *not* be surprised later when they do "emerge -u kde" and find that some packages are corrupted. Patch is below:

--- portage.py.old	2003-05-02 01:49:39.000000000 -0700
+++ portage.py	2003-05-06 11:11:56.000000000 -0700
@@ -1073,7 +1073,17 @@
 					else:
 						#we already have it downloaded, skip.
 						#if our file is bigger than the recorded size, digestcheck should catch it.
-						fetched=2
+						# Unfortunately, for those who don't have a stable/continuous network
+						# connection, this is too late--so check it NOW, while we're connected!
+						mymd5=perform_md5(settings["DISTDIR"]+"/"+myfile)
+						if mymd5 != mydigests[myfile]["md5"]:
+							print "!!! Previously fetched file:",myfile,"MD5 FAILED! Refetching..."
+							os.unlink(settings["DISTDIR"]+"/"+myfile)
+							fetched=0
+						else:
+							print ">>> Previously fetched file:",myfile,"MD5 ;-)"
+							fetched=2
+							break #No need to keep looking for this file, we have it!
 				else:
 					#we don't have the digest file, but the file exists.  Assume it is fully downloaded.
 					fetched=2
@@ -1112,8 +1122,19 @@
 								except:
 									pass
 							continue
-						fetched=2
-						break
+						# File is the correct size--check the MD5 sum for the fetched
+						# file NOW, for those users who don't have a stable/continuous
+						# net connection. This way we have a chance to try to download
+						# from another mirror...
+						mymd5=perform_md5(settings["DISTDIR"]+"/"+myfile)
+						if mymd5 != mydigests[myfile]["md5"]:
+							print "!!! Fetched file:",myfile,"MD5 FAILED! Removing corrupt distfile..."
+							os.unlink(settings["DISTDIR"]+"/"+myfile)
+							fetched=0
+						else:
+							print ">>>",myfile,"MD5 ;-)"
+							fetched=2
+							break
 					except (OSError,IOError),e:
 						fetched=0
 				else:

[Comment 1 Uriah Heep 2003-05-06 14:51:38 UTC]

Oops, forgot to mention this patch is against Portage 2.0.47-r10...

[Comment 2 Marius Mauch (RETIRED) 2003-05-12 14:24:13 UTC]

Created attachment 11836 [details, diff]
patch against portage-2.0.48_pre5

unchanged except for line numbers

[Comment 3 Marius Mauch (RETIRED) 2003-05-12 14:24:38 UTC]

Created attachment 11837 [details, diff]
patch against portage-2.0.48_pre5

unchanged except for line numbers

[Comment 4 Nicholas Jones (RETIRED) 2003-05-14 17:23:24 UTC]

Added in modified, thanks.

[Comment 5 Martin Holzer (RETIRED) 2003-05-22 13:49:33 UTC]

*** Bug 8138 has been marked as a duplicate of this bug. ***