Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 116996 - Doc addition: Moving Portage to its own partition
Summary: Doc addition: Moving Portage to its own partition
Status: RESOLVED WONTFIX
Alias: None
Product: [OLD] Docs-developer
Classification: Unclassified
Component: Portage Documentation (show other bugs)
Hardware: All Linux
: High enhancement (vote)
Assignee: Docs Team
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2005-12-28 08:36 UTC by Peter Hyman
Modified: 2005-12-31 08:27 UTC (History)
1 user (show)

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Peter Hyman 2005-12-28 08:36:37 UTC
There have been many threads on the forums and newsgroups about portage getting held up at 50-51% during the metadata caching. Of all the workarounds suggested, the one I found useful was putting portage on its own partition. This improved performance dramatically for me (from over 5 minutes to under 1) and from a purist's POV, /usr/ is mainly for static non-changing programs. Portage certainly is not that!

I see two areas of the doc which could be updated. 

1) The installation guide could be updated to show the user that he/she _could_ set up a portage partition (a la /boot) and mount /usr/portage appropriately.
2) The portage section of the doc could be updated to help users migrate to a portage-only partition post-install.

I posted this on gentoo-user, and it's just one user's experience. Please feel free to take what you want out of this, or if you would like me to write up something more formal, please let me know.

The following deals with 2) above.
---------------------------------------------------

I'd seen all the threads on the problems with portage getting held up
during emerge metadata. Since around a month ago, I have suffered through
the holdup almost every day. Sometimes, it would take 5 or more minutes!

I read as much as I could on the various forums and ng and wanted to post
MY suggested workaround.

In a nutshell, I found that portage needed to be on its own partition and
had to be backed up and restored onto a clean partition. By doing this
alone, there was no more dreaded 50-51% holdup.

I also found that it did not make much difference whether I used ext2,3 or
reiserfs when running emerge metadata. In fact, reiser was slower by
around 30 seconds (57-ext3 vs 1:30-reiser). This was not surprising since
on the namesys website they note that programs like find won't perform as
well due to its internal organization. Interestingly, if I ran emerge
metadata a second time on reiser, the speed dropped to 45 seconds. I
imagine this is because of directory caching.

On my particular setup, I have one partition that includes portage,
portage overlay, distfiles, and portage tmp dir. I set all these in
make.conf.

Having portage in /usr is kind of a contradiction imho, since most files
there are supposed to be static, not changing all the time. Moving portage
and its related files elsewhere makes everything a log cleaner, and at
least in my case, faster.

From my make.conf. Note, /mnt/src is my portage partition.

PORT_LOGDIR="/var/log/portage"
PORTDIR="/mnt/src/portage"
PORTDIR_OVERLAY="/mnt/src/local/portage" PORTAGE_TMPDIR="/mnt/src/var/tmp"
DISTDIR="/mnt/src/distfiles"

For right now, I have reiser handling /mnt/src, and I'll leave this alone
for a while. Don't know if performance will improve. It _might_ since the
act of syncing will force a read through the entire portage tree which
might make metadata faster

HTH.

Suggestions:

0) You _are_ backed up, right? At least, save the directories you'll be
working on!
1) make a partition. 1+G for /usr/portage. Larger for additional files. I
have 7 gig for my setup. Filesystem is up to you. Just make sure kernel
support is enabled (i.e. reiser) and appropriate support files loaded. 
1a) mount with options noatime
2) copy over existing portage using cp -Rp (preserve times and permissions
otherwise portage will download the entire tree), or tar it and untar it.
Whatever works. Just make sure that make.conf is adjusted and
/etc/make.profile is relinked to the new location.

/etc/make.profile -> /mnt/src/portage/profiles/default-linux/x86/2005.0

This is important otherwise any emerge command will bomb. Your link may
differ depending on your arch. ls -l /etc/make.profile will show what you
need to do. Just substitute your new portage location and add the rest.

3) update make.conf.
4) see if everything works, try emerge -s, emerge sync, metadata. 
5) if it does, you can purge /usr/portage and any other directories you moved over to your dedicated portage partition.
Comment 1 Alec Warner (RETIRED) archtester gentoo-dev Security 2005-12-28 09:16:05 UTC
Portage team doesn't manage the handbook, reassigning to docs and cc'ing dev-portage.

I'd also note Peter, that the cache update problem (50-51) is fixed in 2.1_preX.
Hopefully with the new code this problem will not crop up again.  However the idea of either moving the default /usr/portage to /var or /srv ( which is actually covered by another bug ) is a decent one, as well as putting portage on it's own partition ( especially if you have the cpu cycles to squashfs it ;) ).

Comment 2 Peter Hyman 2005-12-28 09:30:21 UTC
(In reply to comment #1)
> 
> I'd also note Peter, that the cache update problem (50-51) is fixed in
> 2.1_preX.
I am very bad at searching bugzilla. Ask Jakub. I did not see this!

> Hopefully with the new code this problem will not crop up again.  However the
> idea of either moving the default /usr/portage to /var or /srv ( which is
> actually covered by another bug )
Oops, my bugzilla skills are exposed again...

> is a decent one, as well as putting portage
> on it's own partition ( especially if you have the cpu cycles to squashfs it ;)

Actually, regardless, I agree that portage belongs elsewhere. As for squashfs, isn't that RO? How would that help? Well, I guess I was too impatient to wait for a portage update. It seems this issue has come and gone a couple of times. Separating portage into a unique partition will protect against future portage "enhancements". :)

Comment 3 Xavier Neys (RETIRED) gentoo-dev 2005-12-31 08:27:05 UTC
IMO, moving Portage to its own partition is beyond the scope of our handbook.