Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 103664 - anonymous svn is needed (anonsvn/anoncvs)
Summary: anonymous svn is needed (anonsvn/anoncvs)
Status: RESOLVED FIXED
Alias: None
Product: Gentoo Infrastructure
Classification: Unclassified
Component: Git (show other bugs)
Hardware: All Linux
: High normal (vote)
Assignee: Gentoo Infrastructure
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2005-08-24 23:26 UTC by Brian Harring (RETIRED)
Modified: 2006-11-14 04:07 UTC (History)
10 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Brian Harring (RETIRED) gentoo-dev 2005-08-24 23:26:44 UTC
Followup bug to discussion in #-infra; in short, with portage moving to svn, we
need anon svn available in some form or other.

Snapshot'ing of the repository itself has been suggested, but that's not
particularly tenuable for svn imo; snapshot of the co works, but castrates all
of the vcs capabilities.

So... in short.  We kind of need anonymous svn access, especially with the
rewrite underway.  I'm snapshotting checkouts in the meantime, but manual work
sucks :)

Either way, opening the bug for discussion to followup here.
Related note, the CVS component probably should be broadened, or SVN added to
the components
Comment 1 Lance Albertson (RETIRED) gentoo-dev 2005-08-24 23:36:19 UTC
Just going to note that we're planning on detaching viewcvs from our www nodes
and placing it on one box to make it easier to manage from our point of view.
I'm planning on adding svn to viewcvs at that point hopefully. No ETA on that
since this is going to take some work along with the www node rewrite upgrade
I'm doing (which I *AM* working on atm).  

But aside from viewcvs, I think anoncvs/svn access for any development modules
would be a great benefit to our community for developement. I will say that I do
not want to have gentoo-x86 on this server because its simply too resource
hungry and not needed (no advantage). Another note, this would solve most of our
read-only access devs we have (and thus will make lcars happy, which is a good
thing).
Comment 2 Corey Shields 2005-10-05 20:45:27 UTC
this would basically just require running a listening svn, and allowing 
anon-read access where desired (but no write access). 
 
I can get on this sometime.. 
Comment 3 Lance Albertson (RETIRED) gentoo-dev 2005-10-05 21:03:38 UTC
Chat with me before you do that. I had a rough plan on where to put this and cvs
and how to make it work. Just give me a heads up.
Comment 4 Brian Harring (RETIRED) gentoo-dev 2006-01-15 01:34:30 UTC
Moving this request off of irc and to this bug so others can follow it rather then folks not knowing what's up.

So, status update please; where are we at on this?

Deciding if it's going to be done?
Investigating requirements?
Allocating hardware?
Installation?
Testing?

Comments in irc thus far have indicated you've started anoncvs- would like info tagged here also so people can track it.  Please refrain from RSN responses also, just state it as it is please.
Comment 5 Kurt Lieber (RETIRED) gentoo-dev 2006-01-15 08:25:28 UTC
Deciding if it's going to be done? [done]
Investigating requirements? [done]
Allocating hardware? [done]
Installation? [done]
Testing? [in progress]

currently one issue w/ the network where the box is located.  syncing there doesn't happen nearly as fast as on other boxes. (please abstain from "helpful" troubleshooting suggestions like "is there enough ram, disk, etc.")

ETA: RSN. :P

Don't ask for specific dates.  We don't have them.  It will be done when it's done.
Comment 6 Brian Harring (RETIRED) gentoo-dev 2006-01-15 14:57:20 UTC
Which are you referring to?  anoncvs, or anonsvn, or both?
Comment 7 Mark Loeser (RETIRED) gentoo-dev 2006-01-15 23:21:06 UTC
Is anoncvs for gentoo-x86 being included in this?  Not exactly clear if it is, and it would help my ATs greatly.
Comment 8 Kurt Lieber (RETIRED) gentoo-dev 2006-01-16 03:31:39 UTC
(In reply to comment #6)
> Which are you referring to?  anoncvs, or anonsvn, or both?

Both and yes, anoncvs for gentoo-x86 is being included in this.


Comment 9 Kurt Lieber (RETIRED) gentoo-dev 2006-01-16 03:33:04 UTC
Just to be clear, anything with "anon" in the title is going to be r/o -- we have no plans to allow anonymous commits to anything.
Comment 10 Brian Harring (RETIRED) gentoo-dev 2006-01-16 04:04:34 UTC
(In reply to comment #9)
> Just to be clear, anything with "anon" in the title is going to be r/o -- we
> have no plans to allow anonymous commits to anything.
No one sane is after anonymous commits, just anonymous ro access so non minted devs can access the repo, be on the same page as those with the write bit.

So... not a concern.
Comment 11 Patrick Lauer gentoo-dev 2006-01-16 05:54:43 UTC
(In reply to comment #10)
> just anonymous ro access so non minted devs can access the repo, 
> be on the same page as those with the write bit.

An interesting side-effect of anonCVS (although it's a bit hackish and may be totally insane ;-) )is that you can update from anonCVS, then change the repo and commit to the master server. I've heard anecdotal evidence that FreeBSD has used this without big problems, might reduce load on the master CVS box.
Comment 12 Robin Johnson archtester Gentoo Infrastructure gentoo-dev Security 2006-01-16 12:28:02 UTC
patrick: actually FreeBSD takes it a step further than that.
They push out cvsup, which offers the entire CVS tree (with all revision history). You update against it (which is _really_ fast), and then only do commits back to the master server.
Comment 13 Brian Harring (RETIRED) gentoo-dev 2006-01-20 09:30:31 UTC
Kurt, my earlier request in terms of status (where are we at? essentially), haven't stated if that's in reference to cvs, or svn.

Assuming cvs from past questioning, would like to know if that's the case. If so, status of svn?
Comment 14 Kurt Lieber (RETIRED) gentoo-dev 2006-01-20 10:14:26 UTC
Looking at the title of this bug and my comment, I've already answered that question.
Comment 15 Kurt Lieber (RETIRED) gentoo-dev 2006-01-20 10:38:12 UTC
if we set up a temporary r/o account on lark, how would that be used?  Would any sort of access to that account extend beyond gentoo developers?
Comment 16 Brian Harring (RETIRED) gentoo-dev 2006-01-20 12:39:22 UTC
This may not even be needed- from what you said in irc, network issues are resolved, assuming that leaves ongoing testing.

If you're doing testing, leverage the AT's as guinnea pigs for testing- you get the service hammer on, they get anon*.

If that's a no go, then-

(In reply to comment #15)
> if we set up a temporary r/o account on lark, how would that be used?

cronjobbed pull to whatever distribution mechanism is in use; for portage, we'll probably just use tailor to convert the pull into an update to a bzr repo so we don't have to dick around with setting up a full vcs server (can just throw the repo on a webserver and be done with it).  Marien already has this setup, just doesn't have it cronjob'd.  For AT's anon access?  No idea how they were intending on redistributing it so AT's could access it.

> Would any sort of access to that account extend beyond gentoo developers?
case by case.  Again, for portage, it would be gentoo developers using it to for doing automated pulls; for those who need anon* for AT's, I've no clue what they're planning- if I were in their shoes, I'd talk nicely to carpaski to reuse his existing setup.

Halyc0n- comments from the AT side of this?
Comment 17 Mark Loeser (RETIRED) gentoo-dev 2006-01-20 13:58:42 UTC
I'd like my ATs to have access via some account to CVS.  I guess if we got a temporary read-only account I'd have a crontab'd pull to my server and distribute it to my ATs from there, but I'd really rather not have to use my stuff.
Comment 18 Brian Harring (RETIRED) gentoo-dev 2006-01-20 14:09:45 UTC
(In reply to comment #17)
> I'd like my ATs to have access via some account to CVS.  
I'd rather the account was purely used for pulls for redistribution- reasoning being that if it takes (worst case) several months for anon* to go official and for this to be turned off, someone is going to have to juggle a crapload of keys for that account.

That's also not getting into the extra load angle for lark.

> I guess if we got a
> temporary read-only account I'd have a crontab'd pull to my server and
> distribute it to my ATs from there, but I'd really rather not have to use my
> stuff.
Like I said, talk nicely to carpaski- he was toying with the notion already, the only thing he'd have to do is change his method of pulling (no more cvsup).

Mark- offhand, it's a bit extra work for those to setup their own redistribution, but if all that's being done is a pull from infra, this is *really* simple to setup- just takes an ldap record and infra/kingtaco to flip the gentooAccess attribute.

I'd personally prefer to keep this as trivial as possible; keep in mind that any temp ro account for pulling *will* go away, thus the quickest implementation possible should be targetted (imo).
Comment 19 Lance Albertson (RETIRED) gentoo-dev 2006-01-20 14:44:17 UTC
Guys, this will get done soon enough. I'd rather not open up other anoncvs/svn setups because of how close we are here. Just please be patient and it will get completed.
Comment 20 Brian Harring (RETIRED) gentoo-dev 2006-01-20 17:57:37 UTC
(In reply to comment #19)
> Guys, this will get done soon enough. I'd rather not open up other anoncvs/svn
> setups because of how close we are here. Just please be patient and it will get
> completed.
Lance, not trying to be a _total_ jack ass here, but I've gotten the RSN response for the last two months.

If it *is* around the corner, as in a week out, sure, I'll shut up- if it isn't (and I'm betting that to be the case), adding a r/o account so we can pull the data and setup anon ourselves *really* is not that huge of a request.

Only thing infra would have to do in that case is flip the gentooAccess; hell, I could have kingtaco do it.  Y'all wouldn't have to do anything, just would have to sign off on it so we could actually do the legwork.

Yes it's annoying, but we *really* do need this now- I'm not sure if y'all really are understanding that.  For AT's, it's making their job a helluva lot easier (and nuking the potential of being banned from rsync mirrors).  For portage, it's not a matter of making things easier- it's flat out required for non devs to do their work (for stable, 50% of our workforce are non gentoo devs).
Comment 21 Lance Albertson (RETIRED) gentoo-dev 2006-01-20 21:41:46 UTC
(In reply to comment #20)
> (In reply to comment #19)
> > Guys, this will get done soon enough. I'd rather not open up other anoncvs/svn
> > setups because of how close we are here. Just please be patient and it will get
> > completed.
> Lance, not trying to be a _total_ jack ass here, but I've gotten the RSN
> response for the last two months.

I gave you a non-jack ass reply, and now you reply with one. I don't get it.

> Only thing infra would have to do in that case is flip the gentooAccess; hell,
> I could have kingtaco do it.  Y'all wouldn't have to do anything, just would
> have to sign off on it so we could actually do the legwork.

If you or anyone else do this without authorization of either me or klieber, I will revoke your shell access on lark (not cvs, just shell). I strictly forbid anyone with shell access on lark to run such services without our permission. I'm not making this a threat, I'm simply stating that I will not tolerate anyone doing such things without our approval. Just because you have the access to run it doesn't mean its right (I don't care how easy/non-trivial it is). We have always placed strict rules on lark and I'm not making a special exception just because of this.

> Yes it's annoying, but we *really* do need this now- I'm not sure if y'all
> really are understanding that.  For AT's, it's making their job a helluva lot
> easier (and nuking the potential of being banned from rsync mirrors).  For
> portage, it's not a matter of making things easier- it's flat out required for
> non devs to do their work (for stable, 50% of our workforce are non gentoo
> devs).

I do understand that its annoying, just be patient and it will happen. Every time you make some heated comment about this topic, it just makes us even more unmotivated to finish it. So please just let us do our job without the constant pressure. It took me a week or so to finally get the networking problem sorted out. I feel like I'm having to check off every little step we've done to make you feel like there's some sort of progress going on.

So, either cool it or we'll just drop the whole thing. (which I'd rather not do because I know how important this is).
Comment 22 Brian Harring (RETIRED) gentoo-dev 2006-01-21 02:36:03 UTC
(In reply to comment #21)
> (In reply to comment #20)
> > (In reply to comment #19)
> > Only thing infra would have to do in that case is flip the gentooAccess; hell,
> > I could have kingtaco do it.  Y'all wouldn't have to do anything, just would
> > have to sign off on it so we could actually do the legwork.
> run such services without our permission.

Lance, I think you're missing what this request actually is- there isn't any service to run.  We're asking for the same setup you have in place between osprey and lark- no service, just a _simple_ ssh r/o account used for a cronjobbed pull.

If you look above to klieber's request in comment 15, I think he understands what is actually requested here in the interim- comments thus far are in regards to it.

> > Yes it's annoying, but we *really* do need this now- I'm not sure if y'all
> > really are understanding that.  For AT's, it's making their job a helluva lot
> > easier (and nuking the potential of being banned from rsync mirrors).  For
> > portage, it's not a matter of making things easier- it's flat out required for
> > non devs to do their work (for stable, 50% of our workforce are non gentoo
> > devs).
> 
> Every time you make some heated comment about this topic, it just makes us 
> even more unmotivated to finish it. So please just let us do our job without 
> the constant pressure.

Might I suggest an ombuddsman be brought in then?  From my side of things, asking about it results in being attacked in return regardless if I was the person who asked.  Merely *commenting* on it results in being attacked (as was demonstrated today), resulting in folks getting pissy in return.

Perhaps it's time we get an intermediary in here, because it's extremely frustrating to need something from y'all and have the drama everytime some random person (as occured today) asks about anon*; further, it's extremely frustrating having anon* drag out over months when setting up anon* took all of  about a days worth of work when I last did it locally for testing.

That's kind of the core of why people keep asking- hardware allocation is an issue, troubleshooting the setup also, but setting up anon* isn't that complicated- yet it drags out.  What are we missing?  Why does it take so long?

Frankly, at this point nobody cares why it's dragging out- all they care about is ensuring that is finished ASAP, and about the only way they can gauge if any progress is occuring is by asking- double bind, ask, and we get our asses chewed for it and y'all get unmotivated because of percieved harassment over this issue.

Ombuddsman?

> I feel like I'm having to check off every little step we've done to make
> you feel like there's some sort of progress going on.

Lance, the questioning is interproject communication, or random folks just asking.  This is something we *need* now, not later- we *do* need to know what's going on since it's a limiting our ability to do our own work, in some cases drastically (we have no release of rewrite, thus anyone who wants to work on it must track down a dev to feed them every change via tarball).

> So, either cool it or we'll just drop the whole thing. (which I'd rather not 
> do because I know how important this is).

Please avoid threats, they *really* don't accomplish anything but piss people off further, which as you can see, is not really needed- we're all pretty pissed off already as demonstrated by people's behaviour in irc.
Comment 23 Lance Albertson (RETIRED) gentoo-dev 2006-01-21 13:47:20 UTC
Anonymous cvs is now working. Please feel free to test, but don't publically announce anything yet:

cvs -d :pserver:anonymous@anoncvs.gentoo.org:/repositories login
cvs -d :pserver:anonymous@anoncvs.gentoo.org:/repositories co gentoo

It should contain the same information as see in our viewcvs collecton. Please let me know of any problems you may have.

Thanks-
Comment 24 Lance Albertson (RETIRED) gentoo-dev 2006-01-21 13:51:48 UTC
Oh yeah.. its currently updating every 30 minutes.
Comment 25 Torsten Veller (RETIRED) gentoo-dev 2006-01-21 15:19:29 UTC
viewcvs doesn't show userinfo.xml while it is available via anoncvs.

http://www.gentoo.org/cgi-bin/viewcvs.cgi/xml/htdocs/proj/en/devrel/roll-call/userinfo.xml?root=gentoo&rev=1.496&view=log
Comment 26 Lance Albertson (RETIRED) gentoo-dev 2006-01-21 15:26:04 UTC
Doh! I knew I was forgetting something. I've got to head out now, but i'll add an exclude for that (that's really easy). 
Comment 27 Brian Harring (RETIRED) gentoo-dev 2006-01-21 15:35:04 UTC
userinfo.xml, from the web side of things doesn't use birth date at all- could just cleanse that info out unless someone knows of an actual use of it.
Comment 28 Lance Albertson (RETIRED) gentoo-dev 2006-01-22 08:59:26 UTC
(In reply to comment #27)
> userinfo.xml, from the web side of things doesn't use birth date at all- could
> just cleanse that info out unless someone knows of an actual use of it.
 
I'm more concerned with the data you don't get out of the web. I've applied that filter on the syncing scripts.
Comment 29 Kurt Lieber (RETIRED) gentoo-dev 2006-01-22 10:01:11 UTC
anonsvn working as well.  send me an htpasswd crypt if you're a gentoo developer and are willing to test.  (I'm not familiar enough with svn to know if I got all the right bits & pieces.)

Currently, it's only syncing the portage and keychain repositories.  others will be added on request by that repository owner/main dude.  Since I don't know what all is contained in those repositories (passwords, user account info, etc.) it's not for me to decide what does/doesn't get listed.
Comment 30 Lance Albertson (RETIRED) gentoo-dev 2006-01-22 10:11:01 UTC
I'd say its best to just follow what we include on viewcvs. I don't see why we need to be different than it.

Right now these are the repos I'm syncing on viewcvs:

/apache/
/baselayout/
/eselect/
/gentoolkit/
/gentoo-alt/
/gentoo-vdr/
/hardened/
/keychain/
/linux-patches/
/portage/
/sandbox/  
Comment 31 Kurt Lieber (RETIRED) gentoo-dev 2006-01-22 13:30:54 UTC
updated list to match what is available on viewcvs.
Comment 32 Kurt Lieber (RETIRED) gentoo-dev 2006-01-22 14:02:37 UTC
anoncvs consistently hangs mid-way through a cvs co.  This was verified by three different users (including myself) on three different machines.  Spent some time looking at the strace output, but didn't see anything obvious.  I've got better things to do with my sunday.
Comment 33 Mark Loeser (RETIRED) gentoo-dev 2006-01-22 14:45:48 UTC
(In reply to comment #32)
> anoncvs consistently hangs mid-way through a cvs co.  This was verified by
> three different users (including myself) on three different machines.  Spent
> some time looking at the strace output, but didn't see anything obvious.  I've
> got better things to do with my sunday.
> 

Which repo are you checking out?  Did it fail at the same point each time?  I just checked out gentoo-x86 fine.

Can I atleast give the anon information to my ATs so they can use it?
Comment 34 Lance Albertson (RETIRED) gentoo-dev 2006-01-22 14:47:15 UTC
Yeah, you can go ahead and give this info to your ATs. Just make sure they understand we're still testing it and it may not be up 100% of the time.
Comment 35 Kurt Lieber (RETIRED) gentoo-dev 2006-01-22 16:11:23 UTC
was checking out gentoo/xml via cvs when I had the problem.
Comment 36 Xavier Neys (RETIRED) gentoo-dev 2006-01-22 16:21:18 UTC
(In reply to comment #35)
> was checking out gentoo/xml via cvs when I had the problem.

Had the same problem. Had to hit ctrl-C.
Another cvs co would download a few more files and hang again.
cvs update downloaded the whole gentoo module in one go.
Comment 37 Brian Harring (RETIRED) gentoo-dev 2006-01-22 16:30:14 UTC
getting a hang via svn also; seems to only trigger on larger checkouts (all of portage rather then just a branch).
Comment 38 Mike Doty (RETIRED) gentoo-dev 2006-06-13 12:13:12 UTC
progress!

I think we've worked through the network issues, the next step is to secure the services(it's easy to DoS cvs)
Comment 39 Mike Doty (RETIRED) gentoo-dev 2006-06-25 12:33:48 UTC
we probably shouldn't sync the gentoo/ cvs tree.... someone give me a good reason to sync it....
Comment 40 Lance Albertson (RETIRED) gentoo-dev 2006-06-25 16:43:24 UTC
It has the website in it which will help the translator folks who currently depend on a hourly tarball. What is your main concern with that module?
Comment 41 Xavier Neys (RETIRED) gentoo-dev 2006-06-26 02:57:14 UTC
(In reply to comment #39)
> we probably shouldn't sync the gentoo/ cvs tree.... someone give me a good
> reason to sync it....

It's very much in demand. Anyone trying to help with docs or our web sites needs it to run his local copy, to create patches, to generate docs to other formats...
Infra is not the only team that gets those silly questions about anoncvs and when it will be available :)

Comment 42 Curtis Napier (RETIRED) gentoo-dev 2006-06-26 13:57:48 UTC
(In reply to comment #41)
> (In reply to comment #39)
> > we probably shouldn't sync the gentoo/ cvs tree.... someone give me a good
> > reason to sync it....
> 
> It's very much in demand. Anyone trying to help with docs or our web sites
> needs it to run his local copy, to create patches, to generate docs to other
> formats...
> Infra is not the only team that gets those silly questions about anoncvs and
> when it will be available :)
> 

I couldn't agree more. I get about 3 or 4 requests a month to tar up the tree and send it to people. Mostly from people who want to be able to run a local copy of the site when they are off-line (such as docs people like xavier already said but also from regular users who just want to have a local copy).
Comment 43 Brian Harring (RETIRED) gentoo-dev 2006-06-28 20:52:30 UTC
*bump*, status?

If help will speed this along, state it, else an eta would be _really_ lovely ('twas RSN 5 months ago, and RSN 5 months before that :)
Comment 44 Torsten Veller (RETIRED) gentoo-dev 2006-08-29 16:43:36 UTC
userinfo.xml is again available via anoncvs (#25).

BTW: Can the CVS admins remove /var/cvsroot/gentoo/xml/htdocs/proj/en/devrel/Attic/userinfo.xml entirely?  Does it need its own bug?
Comment 45 Robin Johnson archtester Gentoo Infrastructure gentoo-dev Security 2006-08-29 16:50:27 UTC
Tove: why do you want /var/cvsroot/gentoo/xml/htdocs/proj/en/devrel/Attic/userinfo.xml,v removed?
Removal of files is something we are trying very hard to avoid. If there are privacy issues, we can block it from going to viewcvs/anoncvs, but there should be no reasons to actually remove it (unless it had for example somebodies private SSH keys, unprotected, or similar lunacy).
Comment 46 Brian Harring (RETIRED) gentoo-dev 2006-08-29 16:52:49 UTC
(In reply to comment #45)
> Tove: why do you want
> /var/cvsroot/gentoo/xml/htdocs/proj/en/devrel/Attic/userinfo.xml,v removed?
> Removal of files is something we are trying very hard to avoid. If there are
> privacy issues, we can block it from going to viewcvs/anoncvs, but there should
> be no reasons to actually remove it (unless it had for example somebodies
> private SSH keys, unprotected, or similar lunacy).

Arguement about protecting the file was always based around the fact it has the persons birthday.

Why it has the persons birthday, nfc, but that's the only bit of data in it that isn't displayed via the web.
Comment 47 Lance Albertson (RETIRED) gentoo-dev 2006-08-29 17:01:31 UTC
(In reply to comment #44)
> userinfo.xml is again available via anoncvs (#25).
> 
> BTW: Can the CVS admins remove
> /var/cvsroot/gentoo/xml/htdocs/proj/en/devrel/Attic/userinfo.xml entirely? 
> Does it need its own bug?
> 

I thought I added an exclude for that.. Did it get removed? Please add that exclude back. There's no need for that file to be out in its current form with all the personal information in it. I was planning on excluding it anyways.
Comment 48 Brian Harring (RETIRED) gentoo-dev 2006-08-29 17:15:55 UTC
(In reply to comment #47)
> (In reply to comment #44)
> > userinfo.xml is again available via anoncvs (#25).
> > 
> > BTW: Can the CVS admins remove
> > /var/cvsroot/gentoo/xml/htdocs/proj/en/devrel/Attic/userinfo.xml entirely? 
> > Does it need its own bug?
> > 
> 
> I thought I added an exclude for that.. Did it get removed? Please add that
> exclude back. There's no need for that file to be out in its current form with
> all the personal information in it. I was planning on excluding it anyways.

Again, the only extra personal information in it is birthday.  Additionally-
http://sources.gentoo.org/viewcvs.py/gentoo/xml/htdocs/proj/en/metastructure/userinfo.xml 
is just as accessible.

Seems a bit saner to just take care of bug 129355, and punt the damn file from cvs once and for all...
Comment 49 Robin Johnson archtester Gentoo Infrastructure gentoo-dev Security 2006-08-29 18:23:58 UTC
The original reason for adding birthday was to keep track of the specific cases where legal age made a difference in paperwork. IIRC some of the foundation stuff required that the members were sufficently old (eg not a minor).
Comment 50 Robin Johnson archtester Gentoo Infrastructure gentoo-dev Security 2006-08-29 18:25:44 UTC
Let's go ahead and protect all of them.
I'll make a double check list in a moment.
However they should still stay.
Comment 51 Lance Albertson (RETIRED) gentoo-dev 2006-08-29 18:30:15 UTC
(In reply to comment #48)
> http://sources.gentoo.org/viewcvs.py/gentoo/xml/htdocs/proj/en/metastructure/userinfo.xml 
> is just as accessible.

Uhm, that's not the right one :-) Try going to http://sources.gentoo.org/viewcvs.py/gentoo/xml/htdocs/proj/en/devrel/roll-call/userinfo.xml and you'll see that it doesn't work ;-). Look at the last check in of that one you mentioned.
Comment 52 Brian Harring (RETIRED) gentoo-dev 2006-08-29 18:35:12 UTC
(In reply to comment #51)
> (In reply to comment #48)
> > http://sources.gentoo.org/viewcvs.py/gentoo/xml/htdocs/proj/en/metastructure/userinfo.xml 
> > is just as accessible.
> 
> Uhm, that's not the right one :-) Try going to

*cough* look in that data there; it's the original location of the userinfo.xml y'all are worrying about now, same type of data is in it.

So... probably want to restrict that bugger too (still posit just punt the damn files and be done with it, but neh).
Comment 53 Robin Johnson archtester Gentoo Infrastructure gentoo-dev Security 2006-08-29 18:39:11 UTC
All of the following should be restricted:
/var/cvsroot/gentoo/xml/htdocs/proj/en/gdp/tests/Attic/userinfo.xml,v
/var/cvsroot/gentoo/xml/htdocs/proj/en/metastructure/userinfo.xml,v
/var/cvsroot/gentoo/xml/htdocs/proj/en/devrel/Attic/userinfo.xml,v
/var/cvsroot/gentoo/xml/htdocs/proj/en/devrel/roll-call/userinfo.xml,v
Comment 54 Torsten Veller (RETIRED) gentoo-dev 2006-08-30 06:04:03 UTC
(In reply to comment #45)
> If there are privacy issues, we can block it from going to viewcvs/anoncvs,
> but there should be no reasons to actually remove it

Blocking is okay too.

(In reply to comment #53)
> All of the following should be restricted:
> /var/cvsroot/gentoo/xml/htdocs/proj/en/gdp/tests/Attic/userinfo.xml,v
> /var/cvsroot/gentoo/xml/htdocs/proj/en/metastructure/userinfo.xml,v
> /var/cvsroot/gentoo/xml/htdocs/proj/en/devrel/Attic/userinfo.xml,v
> /var/cvsroot/gentoo/xml/htdocs/proj/en/devrel/roll-call/userinfo.xml,v

Thanks
Comment 55 Xavier Neys (RETIRED) gentoo-dev 2006-08-30 14:43:03 UTC
(In reply to comment #47)
> I thought I added an exclude for that.. Did it get removed? Please add that
> exclude back. There's no need for that file to be out in its current form with
> all the personal information in it. I was planning on excluding it anyways.

Please don't. It's sourced by many pages at the moment and I expect it will be sourced by all docs shortly. You have the following options (non-exhaustive list):
1. consider birth dates to be a matter of public record.
2. let devs choose whether they want it published, pretty much the current situation (does not solve history though).
3. script the birth dates out before letting the files out (yeah, I know).
4. if you really need them, fill in the gaps and move them to a secure location.
5. block/kill'em all and start with a brand new roll-call without birth dates.
6. <insert here>

Please leave at least the current roll-call readable.
All previous versions/locations are useless.
I'm only interested in it being available in anoncvs, I don't care about sources.g.o but FYI, the protected version is viewable at http://sources.gentoo.org/viewcvs.py//gentoo/xml/htdocs/proj/en/devrel/roll-call/userinfo.xml?rev=HEAD&content-type=text/plain
Comment 56 Robin Johnson archtester Gentoo Infrastructure gentoo-dev Security 2006-09-15 13:49:15 UTC
infra: could we take a vote here, or some other form of solid decision regarding blocking userinfo.xml?

neysx is correct in that birthdates are a matter of public record. Anybody wanting to find them really doesn't have to do much work - additionally, if they were nefarious, they could have taken a copy of userinfo.xml already - the file has been available for a long time.

If there are no other issues (none that myself, kingtaco, solar are aware of), can we please move forward with opening CVS/SVN to the public?
Comment 57 Torsten Veller (RETIRED) gentoo-dev 2006-09-15 14:26:23 UTC
I'd opt for removing birthdates when userinfo.xml is generated from ldap data.

The additional email addresses should be removed too. I guess the devs weren't asked if they like their addresses being published this way. Also making them public won't make them more reliable as longtime contact data. perl_ldap in its current form (and the way the addresses are stored) will not export them.
Comment 58 Robin Johnson archtester Gentoo Infrastructure gentoo-dev Security 2006-09-15 14:31:53 UTC
tove: the point is that the history of the file contains the old data. while we can wipe out the history, there's strong calls not to do so.
generating it custom from ldap doesn't really come into the equation.
Comment 59 Torsten Veller (RETIRED) gentoo-dev 2006-09-17 03:51:46 UTC
Robian,
if the history should not be wiped and blocking doesn't work reliably for all the access methods to the repo and copies of userinfo.xml in the repo,
the only thing that can be done is protect any new data by not adding it to userinfo.xml.


I'd prefer: punt all userinfo.xmls out of cvs (as blocking isn't wanted and doesn't work reliable) and generate a new one only with the data needed by the mentioned scripts (i guess nick, name, role, gpgkey?). Further data can be found in ldap records.


If you think no action is needed and "birthdates are matter of public record", i'd like to ask robbat2, solar and kingtaco to add your bithdates and an alternative non-gentoo address of yourself to userinfo.xml.
Comment 60 Xavier Neys (RETIRED) gentoo-dev 2006-09-17 04:57:43 UTC
(In reply to comment #59)
> I'd prefer: punt all userinfo.xmls out of cvs

Not an option. userinfo.xml is required to run www.g.o
Comment 61 Torsten Veller (RETIRED) gentoo-dev 2006-09-17 05:31:55 UTC
Xavier, can you please recomment on the whole sentence. What parts are needed to run www.g.o?

BTW: what i would like to see was once your option no. 5.
Comment 62 Xavier Neys (RETIRED) gentoo-dev 2006-09-17 06:08:05 UTC
(In reply to comment #61)
> Xavier, can you please recomment on the whole sentence. What parts are needed
> to run www.g.o?

/proj/en/devrel/roll-call/userinfo.xml is linked from our main page and it has been sourced by all project pages and some doc indexes (overview.xml pages) for years. It is now also used to replace @gentoo.org email addresses of retired devs in documentation credits by the first non-gentoo email address when one is available or by a name without any mailto: link when no other email is available.
 
> BTW: what i would like to see was once your option no. 5.

Option 5. is still valid.
I'd be happy with 1. but IMHO 5. is the only one that can satisfy everyone. Keeping the current history in a safe place should probably be done.

Birthdays are not used at all.
Non-gentoo email addresses are used but not required. Look at the credits in http://www.gentoo.org/doc/en/handbook/handbook-x86.xml to better understand this. Both drobbins and bennyc are credited with their @gentoo.org emails in the doc. Drobbins's other email is used, bennyc's name is not a link.
BTW, using the former way would be trivial if using non-gentoo emails in credits is not wanted.
Comment 63 Robin Johnson archtester Gentoo Infrastructure gentoo-dev Security 2006-09-17 16:13:24 UTC
tove: I've just added my other email and birthdate to the userinfo.xml. I wasn't aware that they were previously missing, as I'm certain at least one userinfo.xml in the past had it, as I did submit it when I joined Gentoo many moons ago.

one of the problems of keeping the data exclusively in LDAP is that LDAP does not give the history that CVS does. It's perfectly fine for representing the current information, but falls short after that.

Let's look at neysx options for a moment.
> 1. consider birth dates to be a matter of public record.
> 2. let devs choose whether they want it published, pretty much the current situation (does not solve history though).
I'm grouping these together, because aside from handling history, they represent the current state of affairs - any items in the file are presently public record.

> 3. script the birth dates out before letting the files out (yeah, I know).
If perl_ldap ever gets used to generate userinfo.xml like is planned (there's a few blips in the way first), this is trivial to do, but loses us ability to track history on birthdates.

> 4. if you really need them, fill in the gaps and move them to a secure
location.
> 5. block/kill'em all and start with a brand new roll-call without birth dates.
the only true way to be secure in this case would probably be an entire subtree of CVS/SVN that isn't replicated, and even that wouldn't be truely sufficent. if you're considering people stealing the information, a developer could take it just as easily as a user. Anybody determined to get the data will probably still manage it, other routes incl social engineering against a dev with access to the data.

The remove-history case would be keeping the current file named and location, and using the cvs tools to actually remove historical revisions in the ,v file AFTER a backup had been made to another location.

Considering them again, my vote is #1 - #2 is effectively the same thing, representing the current state of affairs (aside from some not totally effective blocking - ). If you pick #2, just don't ever put your birthdate into LDAP/userinfo.xml, and expect that if it is ever needed for a valid reason, that you may be blocked until you do provide it - other than that, nobody is forcing you to ADD your data to the existing system.
Comment 64 Torsten Veller (RETIRED) gentoo-dev 2006-09-18 17:03:55 UTC
As the discussion is drifing away from the topic and i am tired and do think we are already repeating ourselves, my last comment:

Only two things:
- "any items in the file are presently public record"
   Blame your tools (and the devs that copy it to other places,) else it wouldn't be public.

- "let devs choose whether they want it published, pretty much the current situation"
   I don't know if it's true, but i guess the birthdates were added by recruiters without enquiry? Especially without knowing that it is/will be a public record.

Thanks.
Comment 65 SpanKY gentoo-dev 2006-09-18 17:13:49 UTC
ignoring the private junk, what happened to having an anoncvs/anonsvn server ?  anoncvs.gentoo.org doesnt work for me
Comment 66 Kurt Lieber (RETIRED) gentoo-dev 2006-09-18 17:19:59 UTC
(In reply to comment #65)
> ignoring the private junk, what happened to having an anoncvs/anonsvn server ? 
> anoncvs.gentoo.org doesnt work for me
> 

both should be (or at least were) working.  instructions are further up, at least for cvs.  (search for pserver)

if anonsvn.g.o isn't working, try anoncvs.g.o as the domain name (but still using the svn client)
Comment 67 SpanKY gentoo-dev 2006-09-18 17:26:57 UTC
those were the instructions i used:
$ cvs -d :pserver:anonymous@anoncvs.gentoo.org:/repositories login
Logging in to :pserver:anonymous@anoncvs.gentoo.org:2401/repositories
CVS password:
cvs [login aborted]: unrecognized auth response from anoncvs.gentoo.org: cvs [pserver aborted]: /repositories: no such repository
Comment 68 Mike Doty (RETIRED) gentoo-dev 2006-09-18 17:48:25 UTC
(In reply to comment #67)
> those were the instructions i used:
> $ cvs -d :pserver:anonymous@anoncvs.gentoo.org:/repositories login
> Logging in to :pserver:anonymous@anoncvs.gentoo.org:2401/repositories
> CVS password:
> cvs [login aborted]: unrecognized auth response from anoncvs.gentoo.org: cvs
> [pserver aborted]: /repositories: no such repository
> 
I posted on core a couple of weeks ago the instructions for both cvs and svn.  s:/repositories:/var/cvsroot: for cvs only.
Comment 69 SpanKY gentoo-dev 2006-09-18 17:52:33 UTC
thanks, that works ... how about this:

$ cvs -d :pserver:anonymous@anoncvs.gentoo.org:/var/cvsroot co -c
server doesn't support gzip-file-contents
Comment 70 Robin Johnson archtester Gentoo Infrastructure gentoo-dev Security 2006-09-18 17:53:50 UTC
Stop using -z.
We wrote a custom patch to reject anybody using -z due to server load reasons.
Comment 71 Robin Johnson archtester Gentoo Infrastructure gentoo-dev Security 2006-09-30 13:29:17 UTC
changing subject to make finding this bug easier.

Kurt & Lance: any further queries/requests on this, esp. with regards to what Torsten was concerned about, or can I submit an article for this coming GWN to announce?
Comment 72 Robin Johnson archtester Gentoo Infrastructure gentoo-dev Security 2006-10-02 13:54:12 UTC
neysx: a few of us have been talking, and we've come up with a potential hybrid solution, but it would need action by you/GDP.

1. You state that you're using the userinfo.xml to handle automated email address replacements. With the exception of that and the userinfo pages, is there ANYTHING else you are using userinfo.xml for?

2. if we block userinfo.xml to anonymous users, it should not affect the web nodes, but if you do have other uses of userinfo.xml above, it would affect anybody using an anoncvs checkout to validate other files. Are you set in this usage pattern - could you perhaps choose a different tack that doesn't use userinfo.xml?

Doing these would enable blocking to be an adequete solution to the userinfo.xml problem case. Basically we would block it to anonymous users, and allow anybody with access to the real CVS to have access to it - as we claim that we trust our developers.
Comment 73 Xavier Neys (RETIRED) gentoo-dev 2006-10-03 01:48:59 UTC
(In reply to comment #72)
> 1. You state that you're using the userinfo.xml to handle automated email
> address replacements. With the exception of that and the userinfo pages, is
> there ANYTHING else you are using userinfo.xml for?

That's the new thing. userinfo.xml has been used for a couple of years on our overview.xml pages and for longer than I have been with Gentoo on project pages.
Up until last week, sourcing a missing file file would have ended in an error 500, not anymore.

> 2. if we block userinfo.xml to anonymous users, it should not affect the web
> nodes, but if you do have other uses of userinfo.xml above, it would affect
> anybody using an anoncvs checkout to validate other files. Are you set in this
> usage pattern - could you perhaps choose a different tack that doesn't use
> userinfo.xml?

Not related to validation. Just the rendering and it's no big deal anymore as nothing will break.

> Doing these would enable blocking to be an adequete solution to the
> userinfo.xml problem case. Basically we would block it to anonymous users, and
> allow anybody with access to the real CVS to have access to it - as we claim
> that we trust our developers.

Fine with me.

FYI, for those who will use anoncvs to render www.g.o on their own box to test GuideXML, write docs, translate..., there would be only 2 differences as far as I can remember:
.Retired devs' gentoo email will be used
.Dev names will be missing from overview.xml files and project pages

I see no reason those minor issues should delay the annoucement of anoncvs to our users.
If there's a strong demand for it, I can easily solve this by writing a simple transform to deliver an expurged version from www.g.o
Comment 74 Robin Johnson archtester Gentoo Infrastructure gentoo-dev Security 2006-10-03 15:59:47 UTC
thanks neysx. now that we have a consensus that blocking is ok, lets just make sure we have it implemented correctly, and get on with this!

folks, is this the complete list of stuff to block?
gentoo-projects/www-redesign/htdocs/proj/en/metastructure/userinfo.xml,v
gentoo-projects/www-redesign/htdocs/proj/en/devrel/roll-call/userinfo.xml,v
gentoo-projects/www-redesign/htdocs/proj/en/devrel/userinfo.xml,v
gentoo/xml/htdocs/proj/en/gdp/tests/Attic/userinfo.xml,v
gentoo/xml/htdocs/proj/en/metastructure/userinfo.xml,v
gentoo/xml/htdocs/proj/en/devrel/Attic/userinfo.xml,v
gentoo/xml/htdocs/proj/en/devrel/roll-call/userinfo.xml,v

Ramereth: 
could you please check my exclude lines on heron in the cvs replication script?
and check viewcvs/sources for the above exclusions as well?
Comment 75 Robin Johnson archtester Gentoo Infrastructure gentoo-dev Security 2006-10-06 15:26:53 UTC
ok, anoncvs is blocking all of these items properly, so I'm going to go ahead with announcing it to the public now.
Comment 76 Robin Johnson archtester Gentoo Infrastructure gentoo-dev Security 2006-11-14 04:07:06 UTC
closing this since anon* is now alive.