Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 564982 - Please create repo for historical git migration
Summary: Please create repo for historical git migration
Status: RESOLVED FIXED
Alias: None
Product: Gentoo Infrastructure
Classification: Unclassified
Component: Git (show other bugs)
Hardware: All Linux
: Normal normal (vote)
Assignee: Gentoo Infrastructure
URL:
Whiteboard:
Keywords:
: 778974 (view as bug list)
Depends on:
Blocks:
 
Reported: 2015-11-06 02:23 UTC by Richard Freeman
Modified: 2021-08-26 17:52 UTC (History)
3 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Richard Freeman gentoo-dev 2015-11-06 02:23:38 UTC
Please create a repo to hold the historical git migration and give me access to push.  I intend to push the draft I have on github.  Nothing prevents us from pushing a forced update to a newer migration if at a future point in time somebody finds a problem in it or comes up with a better approach, but I've yet to hear any complaints.  It would be better to get this hosted on infra, especially while changelogs are down.

I'm not picky on names.  Suggestions ordered by preference:
repo/gentoo-history
repo/gentoo-cvs
repo/gentoo-<date> - date of the migration
proj/git-migration-history

Reproducible: Always
Comment 1 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2015-11-06 07:20:29 UTC
Just to be clear, the bare clone is around 1.63 GiB, though don't know if it grows more after delta resolution.
Comment 2 Richard Freeman gentoo-dev 2015-11-06 12:22:06 UTC
(In reply to Michał Górny from comment #1)
> Just to be clear, the bare clone is around 1.63 GiB, though don't know if it
> grows more after delta resolution.

That is correct.

If there is a concern about overhead hosting this, I also have a bundle of it which could be hosted on mirrors/etc if that makes more sense.  It isn't really any harder to clone from.  It isn't any smaller, but it can be transferred by http/ftp/rsync/etc and trivially mirrored.

I don't really see anybody doing shallow clones of the historical migration, so I don't think there is much downside to not having it hosted as a full repo.

Either way I still need someplace to put it.  Since it won't be routinely modified I can just put it on dev.g.o for infra to grab if desired.
Comment 3 Robin Johnson archtester Gentoo Infrastructure gentoo-dev Security 2015-11-09 02:18:40 UTC
I was originally planning on repo/gentoo/historical as the name, but I hadn't settled it 100%.

My work was already based on your GitHub instance, plus adding the cleanup commits, so that the state of HEAD on the historical repo already matched the initial commit in gentoo.git.

I just never got that part 100% finished with everything else going on.
Comment 4 Richard Freeman gentoo-dev 2015-11-11 01:10:21 UTC
I'd suggest publishing what we have at least for now, especially since we're without changelogs for a while.  The value of the historical migration actually is at its highest right now, and of course nothing prevents us from appending to the historical migration or just forcing a new head if we redo it.
Comment 5 Robin Johnson archtester Gentoo Infrastructure gentoo-dev Security 2016-03-14 02:50:13 UTC
rich0:
I went to use your copy from github, but I noticed that the entire app-backup category is missing.

(I also seem to have misplaced my intermediate conversion working copy).
Comment 6 Robin Johnson archtester Gentoo Infrastructure gentoo-dev Security 2016-04-04 02:34:11 UTC
rich0:
Here's why app-backup was missing, along with what else:
# ./create-git.sh
Initialized empty Git repository in /home/gitmig/git-conversion-tools/git/
ignoring nonexistant alternates source /home/gitmig/git-conversion-tools/final/sound-data
ignoring nonexistant alternates source /home/gitmig/git-conversion-tools/final/app-mobphone
ignoring nonexistant alternates source /home/gitmig/git-conversion-tools/final/app-backup
ignoring nonexistant alternates source /home/gitmig/git-conversion-tools/final/.cvsignore
ignoring nonexistant alternates source /home/gitmig/git-conversion-tools/final/embassy-meme
ignoring nonexistant alternates source /home/gitmig/git-conversion-tools/final/ruby-freedb
ignoring nonexistant alternates source /home/gitmig/git-conversion-tools/final/kpackage
ignoring nonexistant alternates source /home/gitmig/git-conversion-tools/final/ipaddress
ignoring nonexistant alternates source /home/gitmig/git-conversion-tools/final/web-proxy
ignoring nonexistant alternates source /home/gitmig/git-conversion-tools/final/Attic
ignoring nonexistant alternates source /home/gitmig/git-conversion-tools/final/mail-server
All commits loaded.. starting dedup runs
progress  0%:   7888 of 788892 commits
...
Comment 7 Robin Johnson archtester Gentoo Infrastructure gentoo-dev Security 2016-04-04 02:36:24 UTC
And the actual error for each of those:
final/app-backup/log:ERROR: A CVS repository cannot contain both cvs-repo/gentoo-x86/app-backup/metadata.xml,v and cvs-repo/gentoo-x86/app-backup/Attic/metadata.xml,v
final/app-mobphone/log:ERROR: No RCS files found under 'cvs-repo/gentoo-x86'!
final/Attic/log:ERROR: No RCS files found under 'cvs-repo/gentoo-x86'!
final/embassy-meme/log:ERROR: No RCS files found under 'cvs-repo/gentoo-x86'!
final/ipaddress/log:ERROR: No RCS files found under 'cvs-repo/gentoo-x86'!
final/kpackage/log:ERROR: No RCS files found under 'cvs-repo/gentoo-x86'!
final/mail-server/log:ERROR: No RCS files found under 'cvs-repo/gentoo-x86'!
final/ruby-freedb/log:ERROR: No RCS files found under 'cvs-repo/gentoo-x86'!
final/sound-data/log:ERROR: No RCS files found under 'cvs-repo/gentoo-x86'!
final/web-proxy/log:ERROR: No RCS files found under 'cvs-repo/gentoo-x86'!
Comment 8 Robin Johnson archtester Gentoo Infrastructure gentoo-dev Security 2016-04-04 02:41:11 UTC
Ha, I remember the app-backup/Attic case now. It's a magical CVS reserved word, and hanno tried to add a package with that name; and CVS really got confused.
Comment 9 Richard Freeman gentoo-dev 2016-04-04 09:06:03 UTC
(In reply to Robin Johnson from comment #8)
> Ha, I remember the app-backup/Attic case now. It's a magical CVS reserved
> word, and hanno tried to add a package with that name; and CVS really got
> confused.

Seems like the simple solution is to just delete the offending file(s) and pretend they were never there.  I suspect that cvs wouldn't be able to really retrieve them either.
Comment 10 Robin Johnson archtester Gentoo Infrastructure gentoo-dev Security 2016-04-04 19:15:45 UTC
(In reply to Richard Freeman from comment #9)
> (In reply to Robin Johnson from comment #8)
> > Ha, I remember the app-backup/Attic case now. It's a magical CVS reserved
> > word, and hanno tried to add a package with that name; and CVS really got
> > confused.
> 
> Seems like the simple solution is to just delete the offending file(s) and
> pretend they were never there.  I suspect that cvs wouldn't be able to
> really retrieve them either.
I did some tweaks that should let it actually get included.

I see that the script.sh also didn't catch anything that in the real top-level Attic.

I've got a run going that:
- has a fixup for app-backup/Attic -> app-backup/attic
- includes toplevel Attic
- excludes empty toplevel directories.
Comment 11 Kent Fredric (IRC: kent\n) (RETIRED) gentoo-dev 2017-02-01 19:13:18 UTC
Ping.

gentoo/historical.git has been around for a while now, and the remaining issues with CVS migration are still not corrected in it.

The issue is fixed in the new working git repo, but the historical image is still missing app-backup in entirety, which makes archaeology a bit confusing without having to change your tools entirely.


https://gitweb.gentoo.org/repo/gentoo/historical.git/tree/

d---------      app-arch
d---------	app-benchmarks
d---------	app-cdr

https://gitweb.gentoo.org/repo/gentoo.git/tree/

d---------	app-arch
d---------	app-backup
d---------	app-benchmarks
d---------	app-cdr

$ git log --shortstat app-backup | tail -n 100

commit 56bd759df1d0c750a065b8c845e93d5dfa6b549d
Author: Robin H. Johnson <robbat2@gentoo.org>
Date:   Sat Aug 8 13:49:04 2015 -0700

    proj/gentoo: Initial commit

...

 337 files changed, 16497 insertions(+)
Comment 12 Alec Warner (RETIRED) archtester gentoo-dev Security 2021-03-28 18:46:05 UTC
*** Bug 778974 has been marked as a duplicate of this bug. ***
Comment 13 Alec Warner (RETIRED) archtester gentoo-dev Security 2021-08-26 17:52:27 UTC
I believe ulm's new repo resolves this problem.