Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 133455 - Set up gorg on dev.gentoo.org
Summary: Set up gorg on dev.gentoo.org
Status: RESOLVED FIXED
Alias: None
Product: Gentoo Infrastructure
Classification: Unclassified
Component: Dev box issues (show other bugs)
Hardware: All Linux
: High normal
Assignee: Gentoo Infrastructure
URL:
Whiteboard:
Keywords:
Depends on: 137258
Blocks:
  Show dependency tree
 
Reported: 2006-05-16 02:11 UTC by Xavier Neys (RETIRED)
Modified: 2007-05-15 14:16 UTC (History)
3 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
/etc/xml/catalog (catalog,449 bytes, text/plain)
2006-05-25 10:22 UTC, Xavier Neys (RETIRED)
Details
/etc/gorg/gorg.conf (gorg.conf,4.09 KB, text/plain)
2006-10-10 06:12 UTC, Xavier Neys (RETIRED)
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Xavier Neys (RETIRED) gentoo-dev 2006-05-16 02:11:02 UTC
TODO list:
1. rev bump gorg to 0.6.1 (BTW Lance, gorg.{f}cgi were already chmod'ed a+x in the tarball.)
2. emerge gorg & check apache config
3. make pecker sync /dtd /xsl /css like our www nodes
4. optional: make the roll-call available, used e.g. by project pages
5. config gorg.conf, especially "httphost = dev *"
6. edit our xsl to test where it is being run, e.g. to force http://www.gentoo.org/ in front of links to images, adapt menus or use $link instead of @link
Comment 1 Mike Doty (RETIRED) gentoo-dev 2006-05-17 09:09:15 UTC
last I rememeber, we were going to use 0.5 because of the sql dep in later versions.  can that be removed in the 0.6 branch?
Comment 2 Xavier Neys (RETIRED) gentoo-dev 2006-05-18 05:42:54 UTC
Pecker needs 0.6.1 to handle /~username/ files and mysql is neither required nor requested.
Comment 3 Mike Doty (RETIRED) gentoo-dev 2006-05-24 07:15:57 UTC
gorg is bumped on pecker.  neysx, can you be a little bit more verbose about "check apache config"
Comment 4 Xavier Neys (RETIRED) gentoo-dev 2006-05-25 10:21:12 UTC
(In reply to comment #3)
> gorg is bumped on pecker.  neysx, can you be a little bit more verbose about
> "check apache config"

Compare it with loon/kiwk/wren/parakeet. See that fastcgi is enabled and gorg set as a handler for xml files.
BTW, will users be able do disable gorg in their .htaccess with "RemoveHandler .xml"?

Something I forgot: you should edit /etc/xml/catalog to make sure DTDs used as http://www.gentoo.org/dtd/* use local files instead of hitting the web nodes.
It's not only a waste of time but also makes the generated html uncacheable.
Comment 5 Xavier Neys (RETIRED) gentoo-dev 2006-05-25 10:22:49 UTC
Created attachment 87498 [details]
/etc/xml/catalog

Example, used by about all GDP members and on gentoo.neysx.org
Please check the path it rewrites to is where the DTDs are sync'ed
Comment 6 Xavier Neys (RETIRED) gentoo-dev 2006-07-09 11:00:37 UTC
(In reply to comment #0)
> 3. make pecker sync /dtd /xsl /css like our www nodes

Rsync'ing /doc/en/inserts.xml on top of those 3 dirs would save us quite some xsl tweaking. Thanks.
Comment 7 Mike Doty (RETIRED) gentoo-dev 2006-07-09 11:47:13 UTC
all setup but for syncing of css/dtd/xsl.... can't use wget because of 403, guess we'll have to rsync it.... Lance?
Comment 8 Curtis Napier (RETIRED) gentoo-dev 2006-07-24 15:17:41 UTC
http://www.gentoo.org/proj/en/infrastructure/dev-webspace.xml

updated, in cvs. ping me on irc if you want anything else added to it.
Comment 9 Xavier Neys (RETIRED) gentoo-dev 2006-07-25 02:13:47 UTC
> XML uses the same XSL and parser (gorg) from www.gentoo.org.

gorg is not a parser. It applies XSl to XML and sends the result.

> In the meantime you will have to include the full
> path to links residing on www.gentoo.org.
> ie http://www.gentoo.org/doc/en/index.xml
> instead of /doc/en/index.xml. This document will be updated when the issue
> has been addressed.

Plain wrong.
http://www.gentoo.org/doc/en/index.xml
http://www.gentoo.org/doc/en/
/doc/en/index.xml
/doc/en/

Those 4 links should all lead to the same doc on www.g.o. but don't atm.
Look at your own link to xml-guide.xml
I'll fix it when it can be fixed.
IMO, it should just be a note the let devs know it does not work but will eventually work.

> Note: To remove gorg as the .xml handler and use your own custom xsl put the
> following in your .htaccess file in your public_html directory:
> RemoveHandler .xml

Not quite right.
Removing the handler tells apache to send XML files as it does by default.
If you want to send XML files untransformed, possibly with some xsl *to be processed by the client*, then remove the handler.
FYI, xsl on the client side does not work.
Some clients can only process some xsl with some limitations, some clients can't even try.
Comment 10 Curtis Napier (RETIRED) gentoo-dev 2006-07-25 17:06:01 UTC
(In reply to comment #9)
> > XML uses the same XSL and parser (gorg) from www.gentoo.org.
> 
> gorg is not a parser. It applies XSl to XML and sends the result.
> 
> > In the meantime you will have to include the full
> > path to links residing on www.gentoo.org.
> > ie http://www.gentoo.org/doc/en/index.xml
> > instead of /doc/en/index.xml. This document will be updated when the issue
> > has been addressed.
> 
> Plain wrong.
> http://www.gentoo.org/doc/en/index.xml
> http://www.gentoo.org/doc/en/
> /doc/en/index.xml
> /doc/en/
> 
> Those 4 links should all lead to the same doc on www.g.o. but don't atm.
> Look at your own link to xml-guide.xml
> I'll fix it when it can be fixed.
> IMO, it should just be a note the let devs know it does not work but will
> eventually work.
> 
> > Note: To remove gorg as the .xml handler and use your own custom xsl put the
> > following in your .htaccess file in your public_html directory:
> > RemoveHandler .xml
> 
> Not quite right.
> Removing the handler tells apache to send XML files as it does by default.
> If you want to send XML files untransformed, possibly with some xsl *to be
> processed by the client*, then remove the handler.
> FYI, xsl on the client side does not work.
> Some clients can only process some xsl with some limitations, some clients
> can't even try.
> 

Corrected. In CVS. 

Is that better neysx?
Comment 11 Xavier Neys (RETIRED) gentoo-dev 2006-07-27 02:55:08 UTC
(In reply to comment #10)
> Is that better neysx?

I guess I wasn't clear enough. Sorry.
There's no point writing "Include the full path to links residing on www.gentoo.org if you want them to work correctly in your devspace. ie http://www.gentoo.org/doc/en/ instead of /doc/en/" because there's no difference between "http://www.gentoo.org/doc/en/" and "/doc/en/"
Both point to d.g.o.
Both will point to www.g.o once fixed

Have a look at http://dev.gentoo.org/~neysx/test.xml
Comment 12 Curtis Napier (RETIRED) gentoo-dev 2006-07-27 11:39:22 UTC
(In reply to comment #11)
> (In reply to comment #10)
> > Is that better neysx?
> 
> I guess I wasn't clear enough. Sorry.
> There's no point writing "Include the full path to links residing on
> www.gentoo.org if you want them to work correctly in your devspace. ie
> http://www.gentoo.org/doc/en/ instead of /doc/en/" because there's no
> difference between "http://www.gentoo.org/doc/en/" and "/doc/en/"
> Both point to d.g.o.
> Both will point to www.g.o once fixed
> 
> Have a look at http://dev.gentoo.org/~neysx/test.xml
> 


OK, Now I get yeah. I removed the <note> completely. Should be OK now. In cvs.
Comment 13 Lance Albertson (RETIRED) gentoo-dev 2006-08-26 11:07:46 UTC
I believe this is working now, please reopen if thats not the case.
Comment 14 Xavier Neys (RETIRED) gentoo-dev 2006-08-28 08:32:09 UTC
(In reply to comment #13)
> I believe this is working now, please reopen if thats not the case.

(In reply to comment #0)
> TODO list:
> 3. make pecker sync /dtd /xsl /css like our www nodes
> Rsync'ing /doc/en/inserts.xml on top of those 3 dirs would save us quite some
> xsl tweaking. Thanks.

Not done.
I've seen requests to sync all languages, i.e. /doc/*/inserts.xml
I could live without it, but I suppose it's no more trouble to sync them all than to sync the English one.

> 4. optional: make the roll-call available, used e.g. by project pages

Not done.
If the roll-call is not going to be available, I'd like to know.

> 5. config gorg.conf, especially "httphost = dev *"

Not done.

> 6. edit our xsl to test where it is being run, e.g. to force
> http://www.gentoo.org/ in front of links to images, adapt menus or use $link
> instead of @link
 
Not done.
I'll start working on it when previous points are done.
Comment 15 Lance Albertson (RETIRED) gentoo-dev 2006-08-28 12:07:46 UTC
Thanks for the summary. I wasn't sure where we were at. I'll see if i can look into it soon. Try and catch me on irc today if you can.
Comment 16 Xavier Neys (RETIRED) gentoo-dev 2006-10-10 06:12:40 UTC
Created attachment 99282 [details]
/etc/gorg/gorg.conf

Drop-in replacement for /etc/gorg/gorg.conf
Comment 17 Xavier Neys (RETIRED) gentoo-dev 2006-10-10 06:17:29 UTC
3. if rsyncing from pecker is a problem, would pushing to pecker or using anoncvs be an option?

4. Forget the roll-call, it will not bork on missing files anymore

5. gorg.conf, you have one you can use right now

6. I shan't bother until the rest has been done
Comment 18 Robin Johnson archtester Gentoo Infrastructure gentoo-dev Security 2007-03-14 00:47:53 UTC
neysx: what's still pending on here, could you please clarify?
Comment 19 Xavier Neys (RETIRED) gentoo-dev 2007-03-14 14:48:48 UTC
(In reply to comment #18)
> neysx: what's still pending on here, could you please clarify?

afaik, nothing has been done about points 3. and 5.

3. /dtd/*  /css/*  /xsl/* /doc/*/inserts.xml  need to be sync'ed automatically. anoncvs would do just fine if anything else is deemed unsafe.

5. Just drop the attached gorg.conf into /etc/gorg/

Can you make sure gorg-0.6.3 is used? Differences are minute but I'd rather have the same version as on www.g.o
Emerging it is all it takes, no other action is required.

Does the current crontab run `gorg -C` at all? Once or twice a day should be well enough on pecker IMO. (It cleans up gorg's cache).

Thanks for your help.
Comment 20 Robin Johnson archtester Gentoo Infrastructure gentoo-dev Security 2007-05-15 14:16:44 UTC
Ok, all stuff in place now:
3. dtd/ css/ xsl/ doc/en/inserts.xml are updated every 3 hours from anoncvs.
5. gorg.conf in place.
- gorg-0.6.3-r1 installed.
- gorg -C runs twice a day.