Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 139213 - distfiles mirroring should produce an xml report
Summary: distfiles mirroring should produce an xml report
Status: RESOLVED FIXED
Alias: None
Product: Gentoo Infrastructure
Classification: Unclassified
Component: Other (show other bugs)
Hardware: All Other
: High normal (vote)
Assignee: Gentoo Infrastructure
URL: http://dev.gentoo.org/distfile-mirror...
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2006-07-04 11:04 UTC by Zac Medico
Modified: 2008-01-06 11:36 UTC (History)
1 user (show)

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Zac Medico gentoo-dev 2006-07-04 11:04:32 UTC
We are currently using a script called gen-report-xml.py that generates guide XML directly from mirror-dist logs.  It will be nicer if the guide XML is generated by an XSL transformation of an xml report containing the necessary data.
Comment 1 Curtis Napier (RETIRED) gentoo-dev 2006-07-04 17:44:30 UTC
assigning to myself. see bug#118952
Comment 2 Xavier Neys (RETIRED) gentoo-dev 2006-07-05 01:58:51 UTC
14 months ago:

On Sun, 2005-04-24 at 13:14 +0200, Xavier Neys wrote:

> How to do it better?
>
> 1. Generate XML data that is meaningful to your application
> 2. Write, or more likely, have someone write an XSL transform that will turn
> your data into GuideXML.
>
> The benefits are:
>
> 1. You don't need to worry about GuideXML and you can focus on your data
> 2. Others can reuse your data without having to parse GuideXML
> 3. Whenever anything needs to be changed to GuideXML, all XSL's will be fixed
> at the same time and you needn't be involved
> 4. You don't need to change your application to change the look of your reports.

We all agreed, Brian said he had problems chaining XSLs but did not explain any further. He was sent examples of chained pages on www.g.o
He also mentioned he had scripts that generate XML data. Dunno if any of that is still available.

Hth
Comment 3 Curtis Napier (RETIRED) gentoo-dev 2006-07-07 02:40:00 UTC
<@neysx> we'd have to see some xml data and its DTD first

zac, can you provide these files?
Comment 4 Zac Medico gentoo-dev 2006-07-07 18:26:47 UTC
Sure, but I haven't created them yet.  I can do that this weekend though...
Comment 5 Brian Harring (RETIRED) gentoo-dev 2006-07-10 15:34:39 UTC
(In reply to comment #2)
> 14 months ago:
> 
> On Sun, 2005-04-24 at 13:14 +0200, Xavier Neys wrote:
> We all agreed, Brian said he had problems chaining XSLs but did not explain any
> further. He was sent examples of chained pages on www.g.o

www.g.o xsl/web setup != d.g.o xsl/web setup 14 months back. ;)

I took a stab at it, but more importantly had web monkeys working at it- g2boojum specifically iirc.  Consensus was that the chained processing could not be done on the fly, had to do at least part of the translation up front which wasn't working all that well...

Could be on crack, but from what I recall either were migrating to or from gorg, and of the 3 web servers, configuration varied- that was the reason it failed, and the shitty gen_report.xml route was taken.


> He also mentioned he had scripts that generate XML data. Dunno if any of that
> is still available.

No point in trying to find that script- you have it already, gen-report-xml.py .  Intention wasn't to mangle mirror-dist directly; generates way to many seperate streams of logs to try shoving it all into xml then split out everything but a few channels.  Intention was to general an xml feed via raiding the mirror-dist general logs, which is what gen-report-xml does; just have to change the xml it generates so it's sane.
Comment 6 Xavier Neys (RETIRED) gentoo-dev 2006-07-11 01:20:13 UTC
(In reply to comment #5)
> (In reply to comment #2)
> > 14 months ago:
> > 
> > On Sun, 2005-04-24 at 13:14 +0200, Xavier Neys wrote:
> > We all agreed, Brian said he had problems chaining XSLs but did not explain any
> > further. He was sent examples of chained pages on www.g.o
> 
> www.g.o xsl/web setup != d.g.o xsl/web setup 14 months back. ;)

You didn't need that to make it look good.
http://dev.gentoo.org/~neysx/failure.html

You didn't need it either to chain to XSLs. If you can call xsltproc once to apply guide.xsl, you can call it once before to apply foo.xsl to foo.xml to generate GuideXML.

To de done server side, it had to be copied to www.g.o like the devaway already is.

> I took a stab at it, but more importantly had web monkeys working at it-
> g2boojum specifically iirc.  Consensus was that the chained processing could
> not be done on the fly, had to do at least part of the translation up front
> which wasn't working all that well...

There's never been any such consensus. Servers could chain xsl, did chain xsl and still are. You were pointed at some examples of such pages and never asked for more details on how to cut'n'paste two lines.

> Could be on crack, but from what I recall either were migrating to or from
> gorg, and of the 3 web servers, configuration varied- that was the reason it
> failed, and the shitty gen_report.xml route was taken.

I guess you are.
Both axkit and gorg did and do chain xsl in the same way.

> No point in trying to find that script

/me is not searching but when some xml is available and it hopefully is not too complex for me, I'll write an xsl and turn you guys into happy bunnies.
Comment 7 Brian Harring (RETIRED) gentoo-dev 2007-01-19 00:46:26 UTC
pardon the delay... not cc'ed, thus didn't see the response.
(In reply to comment #6)
> (In reply to comment #5)
> > (In reply to comment #2)
> > > 14 months ago:
> > > 
> > > On Sun, 2005-04-24 at 13:14 +0200, Xavier Neys wrote:
> > > We all agreed, Brian said he had problems chaining XSLs but did not explain any
> > > further. He was sent examples of chained pages on www.g.o
> > 
> > www.g.o xsl/web setup != d.g.o xsl/web setup 14 months back. ;)
> 
> You didn't need that to make it look good.
> http://dev.gentoo.org/~neysx/failure.html
> 
> You didn't need it either to chain to XSLs. If you can call xsltproc once to
> apply guide.xsl, you can call it once before to apply foo.xsl to foo.xml to
> generate GuideXML.
I repeated what the issues the web monkeys at the time stated when I asked them to help me translate straight xml into guidexml.

If things have changed, cool- either way, that *was* the reason it went this way, and whats in place now is rather fugly and should be converted to generating straight xml if the presentation side is good to go.

> To de done server side, it had to be copied to www.g.o like the devaway already is.
May have been part of the problem; dev.g.o's setup was different from www.g.o's, which contributed to some of the issues from what I recall.

> > I took a stab at it, but more importantly had web monkeys working at it-
> > g2boojum specifically iirc.  Consensus was that the chained processing could
> > not be done on the fly, had to do at least part of the translation up front
> > which wasn't working all that well...
> 
> There's never been any such consensus. Servers could chain xsl, did chain xsl
> and still are. You were pointed at some examples of such pages and never asked
> for more details on how to cut'n'paste two lines.

Will digg up logs if you would like.  Strongly suspect d.g.o being involved had a hand in it also.

Meanwhile... 

> > Could be on crack, but from what I recall either were migrating to or from
> > gorg, and of the 3 web servers, configuration varied- that was the reason it
> > failed, and the shitty gen_report.xml route was taken.
> 
> I guess you are.
> Both axkit and gorg did and do chain xsl in the same way.
> 
> > No point in trying to find that script
> 
> /me is not searching but when some xml is available and it hopefully is not too
> complex for me, I'll write an xsl and turn you guys into happy bunnies.

Finger pointing is stupid.

Reverting the script to generate xml can be done (either zac can, or just throw it my way).  Feel free to take a stab at xsl translation of it; bit more productive.
Comment 8 Robin Johnson archtester Gentoo Infrastructure gentoo-dev Security 2008-01-06 11:36:48 UTC
Old bug it seems. The live version is on: http://dev.gentoo.org/distfile-mirroring/failure.xml