Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 628098 - Migrate GLEPs outta wiki
Summary: Migrate GLEPs outta wiki
Status: RESOLVED FIXED
Alias: None
Product: Websites
Classification: Unclassified
Component: Wiki (show other bugs)
Hardware: All Linux
: Normal enhancement (vote)
Assignee: Gentoo Wiki Team
URL:
Whiteboard:
Keywords:
Depends on: 630882 631338
Blocks:
  Show dependency tree
 
Reported: 2017-08-17 13:52 UTC by Michał Górny
Modified: 2017-10-17 15:59 UTC (History)
6 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2017-08-17 13:52:32 UTC
I think that we can reasonably agree at this point that moving GLEPs into wiki wasn't such a great idea. I've listed multiple issues with SMW in the -project thread [1]. However, a handful of them specifically apply to the GLEPs, and those are:

1. Lack of proper review platform and merge capability. We end up reinventing the wheel poorly by copying articles and requesting review of that. Then GLEP editors move them back eating history and sometimes resulting in even worse failures like including a temporary subpage in GLEP namespace [2]. Getting a GLEP in place sometimes takes GLEP editors over *a month* during which time there is not even a clear indication that a GLEP has update pending.

2. The wiki form does not really fit GLEPs. Note that RFCs are plain text files. PEPs use reStructuredText which is also pretty close to plain text. MediaWiki GLEPs are only mildly readable as plain text, and sometimes end up referring to external templates or other materials. They are not stand-alone, and they are hard to copy to use outside the wiki.

3. MediaWiki is not very good at accessibility. It's not convenient for anything more complex than quick article edits, and it ends up being real PITA for people with disabilities [3].


At the same time, I believe that our GLEP concept is not bound closely to the wiki, and that we should be able to reasonably move them outta wiki. What I'd like to deploy is a model similar to Python PEPs, that is:

1. All GLEPs are (once again) text files, possibly using rST (since its syntax is very similar to our old-style text files with footnotes).

2. We put those GLEPs in a git repository. The master branch is restricted to GLEP editors; other developers create their own branches when they need to edit GLEPs. GLEP editors simply cherry-pick commits from dev branches to master.

3. Optionally, we may want to include some OpenPGP signatures with GLEPs, similarly to how we do with news items. Usefulness TBD. For example, we could include the signatures of all Council members when the GLEP is approved by the Council.

4. We offer nice HTML rendering of the GLEPs from master branch on projects.g.o.


And I think that covers all the main points. The benefits are pretty clear: we use git with its full power, publish readable stand-alone text files with optional rendering and optionally get some OpenPGP to stamp things. We can easily work on GLEPs (also offline), get them reviewed and merged without losing the history behind them. Everyone can read and write GLEPs, independently of artificial entry barrier created by poorly written software.

Plus, it's nothing really new. It's similar to how GLEPs operated before the wiki -- however, in a much more modern way. More importantly, it's close to how PEPs operate right now which gives it quite a broad testing.


Unless I'm missing something, the migration would require:

a. Setting up the repository and rendering infrastructure.

b. Converting all existing GLEPs into rST (or another markup of choice) -- which shouldn't be a hard task per se.

c. Updating GLEPs 1&2 as appropriate.


[1]:https://archives.gentoo.org/gentoo-project/message/bf6cdaeb6c9c689b32e1f0c55447ca5d
[2]:https://wiki.gentoo.org/wiki/GLEP:73/VerificationAlgo
[3]:https://archives.gentoo.org/gentoo-project/message/01fc23841c98670b2419a8638f97ec0b
Comment 1 William Hubbs gentoo-dev 2017-08-17 13:57:37 UTC
Yes, let's do this. Mediawiki needs to die.
Comment 2 Chris Reffett (RETIRED) gentoo-dev Security 2017-08-17 22:22:50 UTC
Speaking as the only member of the GLEP team (who, I might add, wasn't one of the people who "reasonably agreed" that it wasn't a great idea, or was even consulted about this), you're welcome to do it yourself. I already did the entire conversion _into_ the Wiki and am not particularly interested in migrating everything yet again.
Comment 3 Robin Johnson archtester Gentoo Infrastructure gentoo-dev Security 2017-08-18 21:36:33 UTC
I'd like to point out that GLEPs were originally in rST, and there is ONLY a single GLEP that was in GuideXML.

And we DO still have the rST->HTML rendering tools; the template should be updated with the Tyrian theme & HTML5, but that's optional enough.

Steps I see:
1. Convert CVS history gentoo/xml/htdocs/proj/en/glep to a Git repo.
1.1. This will include the supplemental files from some of the GLEPs.
2. Import changes made to GLEPs after the wiki migration.
3. Convert GLEP4 from XML -> rST.
4. Disallow XML for GLEPs.
5. Profit
Comment 4 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2017-08-18 22:12:54 UTC
Nice to hear the work needing to be done is not that insane.

While it's nice to reuse existing tools, I was actually wondering about trying to use jekyll to render them. After all, they don't seem much different from the thingies we use for the news on www.g.o, so they might just fit in nicely.

I've already installed jekyll locally and I'll attempt some testing when I have time.
Comment 5 Matthew Marchese Gentoo Infrastructure gentoo-dev 2017-08-22 03:37:28 UTC
If you guys proceed as I'm seeing above I'd suggest not creating a new website for the GLEPs (like we have for the devmanual). Put them under Inside Gentoo -> GLEPs in a reverse chronological order or something on www.g.o.

This keeps the number of sites infra maintains lower and increases the ability to maintain it.

www.g.o already has the ability to grab various data from the web (see /bin in the base of the git project).

I'd also suggest Markdown over rST (if this can work with point #3 in OP) for consistency and because it seems to be the most popular markup language at this time; it also enables developer's forks to be readable on most hosting sites (aka GitHub).
Comment 6 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2017-08-22 07:41:04 UTC
(In reply to Matthew Marchese from comment #5)
> If you guys proceed as I'm seeing above I'd suggest not creating a new
> website for the GLEPs (like we have for the devmanual). Put them under
> Inside Gentoo -> GLEPs in a reverse chronological order or something on
> www.g.o.
> 
> This keeps the number of sites infra maintains lower and increases the
> ability to maintain it.
> 
> www.g.o already has the ability to grab various data from the web (see /bin
> in the base of the git project).

That's what I've been thinking of.

> I'd also suggest Markdown over rST (if this can work with point #3 in OP)
> for consistency and because it seems to be the most popular markup language
> at this time; it also enables developer's forks to be readable on most
> hosting sites (aka GitHub).

rST is supported as widely as Markdown. Also, it is more readable and has the advantage of not having to convert everything again.
Comment 7 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2017-09-10 14:35:29 UTC
I've started working on this as requested. What I've done so far:

1. Converted the old GLEP directory from CVS to git @ [1] (master branches is history stripped of generated data and site files, historical includes all),

2. Collected MW GLEPs into git repository using git-remote-mediawiki @ [2] (wiki branch),

3. Prepared some code to parse and display GLEPs on www.g.o [3].

I think the next step would be to decide on how to alter GLEP 1. This brings two questions:

a. Should I integrate the wiki changes into GLEP 1/2 for the matter of preserving history, or just ignore them and go straight for a new version?

b. Which of the original headers we'd like to preserve? The original GLEP 1 list was quite long:

  GLEP: <glep number>
  Title: <glep title>
  Version: <cvs version string>
  Last-Modified: <cvs date string>
  Author: <list of authors' real names and optionally, email addrs>
* Discussions-To: <email address>
  Status: <Draft | Active | Accepted | Deferred | Rejected |
           Final | Replaced>
  Type: <Informational | Standards Track>
* Content-Type: <text/plain | text/x-rst>
* Requires: <glep numbers>
  Created: <date created on, in dd-mmm-yyyy format>
  Post-History: <dates of postings to gentoo-dev>
* Replaces: <glep number>
* Replaced-By: <glep number>

('*' are optional)

The wiki version [5] removed many of them:

   GLEP: <glep number>
   Title: <glep title>
   Author: <list of authors' real names and optionally, email addrs>
   Status: <Draft | Active | Accepted | Deferred | Rejected |
            Final | Replaced>
   Type: <Informational | Standards Track>
 * Requires: <glep numbers>
 * Replaces: <glep number>

(the table also includes 'Editor' and 'Post History' which don't seem to be available for use)

[1]:https://github.com/mgorny/glep-draft
[2]:https://github.com/mgorny/glep-draft/tree/wiki
[3]:https://github.com/mgorny/www-gentoo-org/tree/gleps
[4]:https://github.com/mgorny/glep-draft/blob/master/glep-0001.txt#L211
[5]:https://wiki.gentoo.org/wiki/GLEP:1#GLEP_Header
Comment 8 Robin Johnson archtester Gentoo Infrastructure gentoo-dev Security 2017-09-11 00:20:27 UTC
(In reply to Michał Górny from comment #7)
> I think the next step would be to decide on how to alter GLEP 1. This brings
> two questions:
> 
> a. Should I integrate the wiki changes into GLEP 1/2 for the matter of
> preserving history, or just ignore them and go straight for a new version?
I think integrate it roughly, then to the next version.

> b. Which of the original headers we'd like to preserve? The original GLEP 1
> list was quite long:
Looking at what was dropped, I think it was mostly accidental, esp. since the rest of the text still reference much of it.

>   GLEP: <glep number>
>   Title: <glep title>
>   Author: <list of authors' real names and optionally, email addrs>
>   Status: <Draft | Active | Accepted | Deferred | Rejected |
>            Final | Replaced>
>   Type: <Informational | Standards Track>
> * Requires: <glep numbers>
> * Replaces: <glep number>
These are the ones kept in the wiki format.

> * Discussions-To: <email address>
>   Created: <date created on, in dd-mmm-yyyy format>
>   Post-History: <dates of postings to gentoo-dev>
> * Replaced-By: <glep number>
These were dropped in the Wiki format, but I think are valuable to keep. That date format should change however, I think it was written everywhere in ISO format instead.

>   Version: <cvs version string>
>   Last-Modified: <cvs date string>
Keep in the spec, but move to a section on 'obsolete' headers? These were meant to be filled automatically by keyword expansion, but Git doesn't do that in the same way. I do wonder if a semantic version in the Version field would be helpful.

> * Content-Type: <text/plain | text/x-rst>
Keep, but the ONLY format supported is now "text/x-rst". I'm surprised it didn't list the ill-fated XML format.
Comment 9 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2017-09-11 21:05:22 UTC
Two more ideas:

1. Let's wrap the preamble in '---', so jekyll will parse it automatically for us, it also seems to make GitHub happier,

2. Let's rename the files to .rst.

Both things done, people can get a pretty decent rendering already at GitHub, e.g. [1].

[1]:https://github.com/mgorny/glep-draft/blob/preamble-test/glep-0001.rst
Comment 10 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2017-10-12 21:25:18 UTC
So the migration is progressing well. Most notably:

- the repositories are set up and working, all GLEPs have been moved,

- www.g.o seems to be serving GLEPs correctly,

- docutils-glep have seen a new release,

- I've added a migration warnings to wiki GLEPs.


Still in TODO:

- radhermit needs to merge gentoo-syntax updates and release them,

- I want to look into providing redirects from old glep.gentoo.org/... addresses. Or do we want that?
Comment 11 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2017-10-17 15:59:29 UTC
Ok, I think it's working well enough to consider this bug fixed.