149979 2006-10-03 11:23 0000 emerge-delta-webrsync: patch to fix false reports of bzip2 incompatibility due to wrong verification path; unnecessary fetches; deprecate use of emerge 2006-10-14 08:42:20 0000 1 1 1 Unclassified Portage Development Tools unspecified All Linux RESOLVED FIXED P2 normal --- 1 kdd@gmx.net zmedico@gentoo.org betelgeuse@gentoo.org kdd@gmx.net 2006-10-03 11:23:37 0000 Hi, I noticed that /usr/bin/emerge-delta-webrsync always reports differences between the distributed file portage-YYYYMMDD.tar.bz2.md5sum from the mirrors and the checksum of the compressed tarball generated by the script. The script claims that this is due to differences between bzip2 versions and then creates a new *.md5sum file. Comparing this new created checksum file with the file from the mirrors the reported checksum is identical and even the verification of the new created compressed tarball against the distributed *.gpgsig from the mirrors is sucessful. This false report about a not matching checksum of the compressed tarball is due to the scipt assuming a wrong current working directory when calling the function verify_md5_file() without giving the second argument for the location of the original *.md5sum. You can check this with the following debug code: diff -Naur orig/emerge-delta-webrsync debug/emerge-delta-webrsync --- orig/emerge-delta-webrsync 2006-09-28 10:29:52.000000000 +0000 +++ debug/emerge-delta-webrsync 2006-10-03 13:27:49.000000000 +0000 @@ -253,6 +253,10 @@ cd "$(dirname ${FILE})" FILE="$(basename ${FILE})" fi + ### DEBUG snippet BEGIN ### + echo "DEBUG INFO: Verifying ${FILE} against ${MD5_LOC} " + echo "DEBUG INFO: Current working directory is $(pwd)" + ### DEBUG snippet END ### if eval "$md5_com"; then [ -n "${CUR}" ] && cd "${CUR}" return 0 Then you can see that the working directory changed from ${DISTDIR} to ${TEMPDIR} for the creation of the new tarball and the location of the tarball and the distributed checksum file differs. I've included a diff for patching the original file emerge-delta-webrsync in the version as emerged from me on 2006-09-29 in the package app-portage/emerge-delta-webrsync-3.5.1 (I only found a cvs/changelog for the ebuild on the web, but not for the app, so I don't know if there are already changes in a development version. The diff is for the file that is installed to /usr/bin/emerge-delta-webrsync). The patch additionally has changes for 2 different issues: a.) emerge is called by the original file with a deprecate variant for metadata as reported in other bugreport(s). The solution http://bugs.gentoo.org/attachment.cgi?id=93681 attached to Bug #127021 http://bugs.gentoo.org/show_bug.cgi?id=127021 seems more well thougt out as my cosmetic addition of "--" to metadata. b.) emerge-delta-webrsync is trying to fetch the *.umd5sum for the existing base tarball even if the checksum file already exist in the distfiles directory. When I'm using this script offline on a machine without a network connection the original script fails even if all necessary files are provided in the distfiles directory. Added a check for the existence of the file before the attempt to fetch the file from the mirrors. diff -Naur orig/emerge-delta-webrsync modified-r1/emerge-delta-webrsync --- orig/emerge-delta-webrsync 2006-09-28 10:29:52.000000000 +0000 +++ modified-r1/emerge-delta-webrsync 2006-09-30 12:09:41.000000000 +0000 @@ -218,7 +218,7 @@ rm -rf portage fi echo "transferring metadata/cache" - emerge metadata + emerge --metadata } fetch_from_mirrors() { @@ -410,7 +410,7 @@ echo "thus, bailing (sorry)" exit 5 else - if ! fetch_from_mirrors "/snapshots/portage-${final_date}.tar.bz2.umd5sum" "portage-${final_date}.tar.bz2.umd5sum"; then + if [[ ! -e portage-${final_date}.tar.bz2.umd5sum ]] && ! fetch_from_mirrors "/snapshots/portage-${final_date}.tar.bz2.umd5sum" "portage-${final_date}.tar.bz2.umd5sum"; then if ! fetch_from_mirrors "/snapshots/portage-${final_date}.tar.bz2.umd5sum" "portage-${final_date}.tar.bz2.umd5sum"; then echo "couldn't grab umd5sum (uncompressed md5sum) for ${final_date}." echo "can't compensate for bzip2 version differences iow." @@ -462,7 +462,7 @@ echo "verifying generated tarball" -if ! verify_md5_file "${TEMPDIR}/portage-${final_date}.tar.bz2"; then +if ! verify_md5_file "${TEMPDIR}/portage-${final_date}.tar.bz2" "${DISTDIR}/portage-${final_date}.tar.bz2.md5sum"; then if [[ -z $verified ]]; then echo "couldn't verify the generated tarball. bug, most likely." exit 5 Best regards Klaus kdd@gmx.net 2006-10-03 11:41:20 0000 Created an attachment (id=98717) Patch to fix wrong verification path and other issues Added a diff file named emerge-delta-webrsync.diff.tar.bz2 as a patch. The listing of the patch in the description field has a wrong text formatting due to false line feeds. Sorry for that. kdd@gmx.net 2006-10-03 12:04:38 0000 Created an attachment (id=98719) Patch as uncompressed textfile emerge-delta-webrsync.diff Submitted the unmodified patch as text/plain diff file so everyone can view the diff in a browser (hopefully) betelgeuse@gentoo.org 2006-10-14 08:40:50 0000 Fixed in 3.5.1-r1 with permission from zmedico. 98717 2006-10-03 11:41 0000 Patch (*.tar.bz2) to fix wrong verification path and other issues emerge-delta-webrsync.diff.tar.bz2 text/plain QlpoOTFBWSZTWQ+ldvsAAdJ/hsowAEB17//4RyNcCv/v3/oAAIAIAAhAAr2x3dLu5TsGRKm9CYU8 mhtKNB6TIMg9QeoDTI0MEElGhBJ4UNGg0AAAAA0HqA9QG1JTxNTU080UB6IAAAAAAAZBJEUxNPRJ ppkMmjQaAAMgDQADzy3n/gAosCEERmwTmoQiSYQDoctDB1Dp5nYmYc9mbHmZBzts8G7oQvhPM7tJ VujPVKN8XtmiV1VlPGhfhGk/ZYUOyD2z6ah2G24xYm78/byYhbL4e0ZXTPcNIkKZZp0a5IWb2Hij 5o5Xnxs8QfqKUVyzTg1Bn5Ext8tBQBVBFRSYEYVcXKHq7kgcEtEI2nBgkVVtCOUEIsulN1JiAmfC 5Xt0TZ2/P9wceqha96KEwX7YpnmYXKV0usLNQOAXzuyLEVnJBpycvnZCMenSGIT1liqk+uUtJ3gd JXl9T7bX5VkGlTJmrKp4umnxgmKmLeIVFBMnGxl/nYOYVS2VnPXKxV4B4l7OZw9aDsJFsY4UjTPg OI5gt5QMnG4lm6UhiEYKKFjXFEdIu5rIHQh/IIdQi7XfgCeYUoRGFYCivhI00U/tGcUDyuKFkoWB VJNhFZEGiB8KGXyNTReA3Qg0RCxy9dAMYnNpUPHpZSmCdNJmUHWWJPIsaK2QT04IBJfrmGkYQyJl UEmYTCtJ5BIzajmmrLDC1Zi0yQ+1GmXbdD+NCgpSLhAnaBjSjSyDXC8nBrNKIWuUEmIp6GlYqQVZ MUaWTxo4kTVQXyZy6VFqAYEiessqyWSNGJMy2VCQUFbiNExYLhEauw4Kct6uiG59RCa4E6HyIdIi EGrqWvhxPsUsK9aKad4RJDFReYHYSpDqFMkR1RGiQCIDguVJayoQn0BVRiyY4lLX8IsCuqrcVChW ClSgaQLIYUJL/F3JFOFCQD6V2+w= 98719 2006-10-03 12:04 0000 Patch as uncompressed textfile emerge-delta-webrsync.diff emerge-delta-webrsync.diff text/plain ZGlmZiAtTmF1ciBvcmlnL2VtZXJnZS1kZWx0YS13ZWJyc3luYyBtb2RpZmllZC1yMS9lbWVyZ2Ut ZGVsdGEtd2VicnN5bmMKLS0tIG9yaWcvZW1lcmdlLWRlbHRhLXdlYnJzeW5jCTIwMDYtMDktMjgg MTA6Mjk6NTIuMDAwMDAwMDAwICswMDAwCisrKyBtb2RpZmllZC1yMS9lbWVyZ2UtZGVsdGEtd2Vi cnN5bmMJMjAwNi0wOS0zMCAxMjowOTo0MS4wMDAwMDAwMDAgKzAwMDAKQEAgLTIxOCw3ICsyMTgs NyBAQAogCQlybSAtcmYgcG9ydGFnZQogCWZpCiAJZWNobyAidHJhbnNmZXJyaW5nIG1ldGFkYXRh L2NhY2hlIgotCWVtZXJnZSBtZXRhZGF0YQorCWVtZXJnZSAtLW1ldGFkYXRhCiB9CiAKIGZldGNo X2Zyb21fbWlycm9ycygpIHsKQEAgLTQxMCw3ICs0MTAsNyBAQAogCWVjaG8gInRodXMsIGJhaWxp bmcgKHNvcnJ5KSIKIAlleGl0IDUKIGVsc2UKLQlpZiAhIGZldGNoX2Zyb21fbWlycm9ycyAiL3Nu YXBzaG90cy9wb3J0YWdlLSR7ZmluYWxfZGF0ZX0udGFyLmJ6Mi51bWQ1c3VtIiAicG9ydGFnZS0k e2ZpbmFsX2RhdGV9LnRhci5iejIudW1kNXN1bSI7IHRoZW4KKwlpZiBbWyAhIC1lIHBvcnRhZ2Ut JHtmaW5hbF9kYXRlfS50YXIuYnoyLnVtZDVzdW0gXV0gJiYgISBmZXRjaF9mcm9tX21pcnJvcnMg Ii9zbmFwc2hvdHMvcG9ydGFnZS0ke2ZpbmFsX2RhdGV9LnRhci5iejIudW1kNXN1bSIgInBvcnRh Z2UtJHtmaW5hbF9kYXRlfS50YXIuYnoyLnVtZDVzdW0iOyB0aGVuCiAJCWlmICEgZmV0Y2hfZnJv bV9taXJyb3JzICIvc25hcHNob3RzL3BvcnRhZ2UtJHtmaW5hbF9kYXRlfS50YXIuYnoyLnVtZDVz dW0iICJwb3J0YWdlLSR7ZmluYWxfZGF0ZX0udGFyLmJ6Mi51bWQ1c3VtIjsgdGhlbgogCQkJZWNo byAiY291bGRuJ3QgZ3JhYiB1bWQ1c3VtICh1bmNvbXByZXNzZWQgbWQ1c3VtKSBmb3IgJHtmaW5h bF9kYXRlfS4iCiAJCQllY2hvICJjYW4ndCBjb21wZW5zYXRlIGZvciBiemlwMiB2ZXJzaW9uIGRp ZmZlcmVuY2VzIGlvdy4iCkBAIC00NjIsNyArNDYyLDcgQEAKIAogZWNobyAidmVyaWZ5aW5nIGdl bmVyYXRlZCB0YXJiYWxsIgogCi1pZiAhIHZlcmlmeV9tZDVfZmlsZSAiJHtURU1QRElSfS9wb3J0 YWdlLSR7ZmluYWxfZGF0ZX0udGFyLmJ6MiI7IHRoZW4KK2lmICEgdmVyaWZ5X21kNV9maWxlICIk e1RFTVBESVJ9L3BvcnRhZ2UtJHtmaW5hbF9kYXRlfS50YXIuYnoyIiAiJHtESVNURElSfS9wb3J0 YWdlLSR7ZmluYWxfZGF0ZX0udGFyLmJ6Mi5tZDVzdW0iOyB0aGVuCiAJaWYgW1sgLXogJHZlcmlm aWVkIF1dOyB0aGVuCiAJCWVjaG8gImNvdWxkbid0IHZlcmlmeSB0aGUgZ2VuZXJhdGVkIHRhcmJh bGwuICBidWcsIG1vc3QgbGlrZWx5LiIKIAkJZXhpdCA1Cg==