Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 54150 - A way to make catalyst a bit faster?
Summary: A way to make catalyst a bit faster?
Status: RESOLVED INVALID
Alias: None
Product: Gentoo Hosted Projects
Classification: Unclassified
Component: Catalyst (show other bugs)
Hardware: All All
: High enhancement (vote)
Assignee: John Davis (zhen) (RETIRED)
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2004-06-16 18:11 UTC by Marc Doughty
Modified: 2004-08-14 06:27 UTC (History)
0 users

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Marc Doughty 2004-06-16 18:11:07 UTC
I think I have an idea to speed up snapshot creation in catalyst. Right now we're making a copy of /usr/portage and excluding a few directories. I think that if you  made a file from a list of /usr/portage, snipped the list items that would be excluded, and tarred items on the list you'd have a 'zero-copy' snapshotting system. Also, I haven't benchmarked it, but I think that if you reduce the 'blocking factor' in tar (defaults to 10KB) you could speed up the snapshotting process (less 'null-padding' for tar to inject and bzip2 to compress).

Reproducible: Always
Steps to Reproduce:
1. 'catalyst -f specs/snap.spec'
2. wait for a full copy of $PORTDIR
3. wait for the copy to get tbz2'd

Actual Results:  
I had time to smoke a cigarette.

Expected Results:  
I would have only enough time to pour another drink.

I've been thinking about a visual tool for catalyst, it could have a lot of
potential. Right now I've got a desk-full of flowcharts for making uber-enhanced
stages. Catalyst has a distinctly 'workflow' feel, and it would be nice to have
a gentoo utility that let you handle all it's features
Comment 1 stephane germain 2004-06-18 21:09:19 UTC
add a switch too keep the working directory when you restart the creation of a livecd (or whatever) following a build error.
Comment 2 John Davis (zhen) (RETIRED) gentoo-dev 2004-08-02 17:00:49 UTC
good idea, but i think that it will cause more problems than it is worth. for example, every time the package list is changed, the snapshot will have to be recreated.

the second comment is being worked on as we speak. I am messing around with autoresuming.

Comment 3 Marc Doughty 2004-08-13 21:32:21 UTC
Well I don't know how other folks use catalyst; I use it to build my own stage3 systems for testing stuff (mostly pre-stage1 stuff, like NPTL when it wasn't built-in) and whipping-up current boot-CDs.

I don't see how it would cause any problems, and it would save several minutes off each snapshotting.

Maybe I'm missing something, but when you 'snap' something, it packs up the portage tree 'the wrong way' for me right now, whereas if you do the kung-fu I described, it takes less time and resources, with the same result.
Comment 4 John Davis (zhen) (RETIRED) gentoo-dev 2004-08-14 06:27:41 UTC
try 1.9.0 in portage (it is ~arch masked). i cleaned up the snapshotting process a bit. after the first time, the time it takes to snapshot should be much improved. i didn't use the method that you recommended (i just tweaked rsync), but it should help. let me know what you think.