Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 57210 - repoman's xmllint causes false positives on flakey connections or when no net access
Summary: repoman's xmllint causes false positives on flakey connections or when no net...
Status: RESOLVED FIXED
Alias: None
Product: Portage Development
Classification: Unclassified
Component: Repoman (show other bugs)
Hardware: All All
: High normal (vote)
Assignee: Portage team
URL:
Whiteboard:
Keywords: InVCS
: 41831 57540 (view as bug list)
Depends on:
Blocks:
 
Reported: 2004-07-15 11:38 UTC by Ciaran McCreesh
Modified: 2004-08-11 16:40 UTC (History)
2 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
repoman-xmllint-cache.patch (patch,10.21 KB, patch)
2004-07-16 00:55 UTC, Brian Harring (RETIRED)
Details | Diff
repoman-xmllint-cache.patch (patch,410 bytes, patch)
2004-07-18 12:47 UTC, Brian Harring (RETIRED)
Details | Diff
repoman-xmllint.patch (patch,11.14 KB, patch)
2004-07-19 12:10 UTC, Brian Harring (RETIRED)
Details | Diff
repoman-dtd.patch , updated to cvs as of 06/29/04. (repoman-dtd.patch,12.85 KB, patch)
2004-07-29 08:52 UTC, Brian Harring (RETIRED)
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Ciaran McCreesh 2004-07-15 11:38:44 UTC
If xmllint can't get the DTD file (www.gentoo.org is timing out 50% of the time for me right now), it'll generate a duff metadata.bad error.

Adding in a print after the xmllint call shows:

RepoMan scours the neighborhood...
mystatus is  (1024, '/home/users/ciaranm/cvs/gentoo-x86/app-vim/genutils/metadata.xml:4: element herd: validity error : No declaration for element herd\n<herd>vim</herd>\n                ^\n/home/users/ciaranm/cvs/gentoo-x86/app-vim/genutils/metadata.xml:6: element email: validity error : No declaration for element email\n  <email>ciaranm@gentoo.org</email>\n                                   ^\n/home/users/ciaranm/cvs/gentoo-x86/app-vim/genutils/metadata.xml:7: element name: validity error : No declaration for element name\n  <name>Ciaran McCreesh</name>\n                              ^\n/home/users/ciaranm/cvs/gentoo-x86/app-vim/genutils/metadata.xml:8: element maintainer: validity error : No declaration for element maintainer\n</maintainer>\n             ^\n/home/users/ciaranm/cvs/gentoo-x86/app-vim/genutils/metadata.xml:9: element pkgmetadata: validity error : No declaration for element pkgmetadata\n</pkgmetadata>\n              ^\n<?xml version="1.0" encoding="UTF-8"?>\n<!DOCTYPE pkgmetadata SYSTEM "http://www.gentoo.org/dtd/metadata.dtd">\n<pkgmetadata>\n<herd>vim</herd>\n<maintainer>\n  <email>ciaranm@gentoo.org</email>\n  <name>Ciaran McCreesh</name>\n</maintainer>\n</pkgmetadata>')

  metadata.bad         1
   app-vim/genutils/metadata.xml


Looks like xmllint just marks everything invalid if it can't get the DTD.
Comment 1 Brian Harring (RETIRED) gentoo-dev 2004-07-16 00:55:55 UTC
Created attachment 35528 [details, diff]
repoman-xmllint-cache.patch

Split against .51_pre13, this cache's metadata.dtd locally at
/var/cache/edb/metadata.dtd .

I went this route, rather then checking for the return code for IO error since
it will speed up repoman a bit- accessing the file locally rather then having
to fetch it.  If the cache'd copy of metadata.dtd is over a week old, repoman
fetches a new copy.

Aside from making it look prettier, the only issue I can think of w/ this patch
is the lack of a lockfile when fetching a new copy of metadata.dtd; strikes me
as this would be better placed into the fetch function, rather then adding the
support explicitly to repoman.

While the majority of this patch is contained in repoman, I did extend fetch
slightly- added optional try_mirrors=1, and use_existing=1 to fetch's
defintion.

try_mirrors controls whether the GENTOO_MIRRORS should be used.  use_existing
is  for controlling whether to try to reuse the existing file (eg, get the last
half of the file), or just plain start from scratch.
Comment 2 Brian Harring (RETIRED) gentoo-dev 2004-07-18 12:47:38 UTC
Created attachment 35707 [details, diff]
repoman-xmllint-cache.patch

Minor one line fix; w/ the fetch function modification, fetchonly in my patch
is abided by a bit stricter- eg, no point in loading up the md5's, since
fetchonly blocks them (not my modification) from actually being used w/in the
fetch function.

The one line fix (well, 2) was correcting a bad emerge call to fetch, setting
fetchonly to enabled.  No point for that from what I can see.
Comment 3 Brian Harring (RETIRED) gentoo-dev 2004-07-18 20:33:16 UTC
*** Bug 57540 has been marked as a duplicate of this bug. ***
Comment 4 Ciaran McCreesh 2004-07-19 11:24:44 UTC
I think the second patch you attached is duff... All I get is five lines of text.
Comment 5 Brian Harring (RETIRED) gentoo-dev 2004-07-19 12:10:38 UTC
Created attachment 35765 [details, diff]
repoman-xmllint.patch

Gah.   Here is the actual patch, not a crappy 'blar differs'.
Comment 6 Ciaran McCreesh 2004-07-19 12:21:04 UTC
Seems to work.
Comment 7 Travis Tilley (RETIRED) gentoo-dev 2004-07-19 16:43:58 UTC
ayanami emul-linux-x86-qtlibs # cat Manifest | grep metadata.xml
MD5 3d107142a3ec13f947ea538fdfe0f517 metadata.xml 290
ayanami emul-linux-x86-qtlibs # md5sum metadata.xml
3d107142a3ec13f947ea538fdfe0f517  metadata.xml

hmm... why arent we making use of the manifest in deciding whether or not metadata.xml needs to be checked? if the md5sum matches that in the manifest, obviously somebody checked it already...

also, why isnt metadata.dtd in the portage tree itself? it seems like that would completely remove the need to fetch it seperately, and would prevent it from ever getting to be a week out of date...

though personally, i dont care as long as i dont have to edit repoman anymore just to get it to work (well, almost work... sometimes... kinda, but not really. ok, it doesnt really work right no matter what i do, but i'd like it to at least pretend)
Comment 8 Brian Harring (RETIRED) gentoo-dev 2004-07-29 08:52:15 UTC
Created attachment 36405 [details, diff]
repoman-dtd.patch , updated to cvs as of 06/29/04.

Ran it past genone, cleaned up the fetch modifications.
Comment 9 Brian Harring (RETIRED) gentoo-dev 2004-08-02 18:31:21 UTC
Commited a simplified version.
Down the line, I'll move the fetch modification in, as is I want repoman's fix in cvs.
Comment 10 Brian Harring (RETIRED) gentoo-dev 2004-08-11 14:42:54 UTC
This is in .51 now.
Comment 11 Brian Harring (RETIRED) gentoo-dev 2004-08-11 14:43:24 UTC
*** Bug 41831 has been marked as a duplicate of this bug. ***
Comment 12 Brian Harring (RETIRED) gentoo-dev 2004-08-11 16:40:54 UTC
Bah.  Thought I resolved it fixed...