Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 89412 - atomic actions for portage to parallelize emerge (future plans)
Summary: atomic actions for portage to parallelize emerge (future plans)
Status: RESOLVED DUPLICATE of bug 1661
Alias: None
Product: Portage Development
Classification: Unclassified
Component: Conceptual/Abstract Ideas (show other bugs)
Hardware: All All
: High normal (vote)
Assignee: Portage team
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2005-04-17 06:58 UTC by Thomas Bettler
Modified: 2005-07-17 13:06 UTC (History)
0 users

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Thomas Bettler 2005-04-17 06:58:53 UTC
Portage should have a better conception.

The few actions for each ebuild should consist of atomic actions.
- Fetch
- Unpack
- Compile
- Install
- Collision-protect (optional)
- Merge
- Config
(or whatever it might look like)

Merging multiple packages meens playing with dependencies. The dependencies should be calculated on the actions.
examples to explain:
- Fetching: always allowed
- Unpacking: unpack x packages to the queue
- Compiling: compile x packages parallel - if dependencies are merged
- Install: always allowed
- Merge: always / or maybe restriction: not parallel
- Config: maybe on a seperate terminal, that admins easily track all the important messages...

These atomic actions could be handled in a continuously modular way on multiple queues. Locking and Unlocking needs to be handled clearly.
examples:
- qt and xorg-x11 could not be compiled together (but fetched, unpacked)
- openoffice and gnomemeeting could be comipled parallel

Advantages:
- distcc would behave far better (even incl. casas of -j1)
- installing packages on a computerfarm would be a picknick
- parallel fetching / merging resolved
- maybe better load balance of diskaccess and cpu
- would make gentoo a favorite distro for compiling reasons

Disadvantages:
- Modular rewrite of portage with great effor needed (so no earlier than in portage 4.0)
- The dependencies of the packages need to be handled while running

New features
Pro and Con: (would be great but needs lot of work and conception)
- emerge.log needs new format to idetify the atomic actions, 
but parallel merges could be better tracked than today
- a new feature "load logger" could maybe be integrated, calculating the GUs used to compile
serving as a base for stats on packages compiling (might be tricky to integrate it into distcc)
- based on stats the progress could be estimated.
Comment 1 Jason Stubbs (RETIRED) gentoo-dev 2005-04-17 07:04:05 UTC
No earlier that portage-4.0? Looking at it for portage-2.2 or whatever it is that comes after 2.1.

*** This bug has been marked as a duplicate of 1661 ***
Comment 2 Thomas Bettler 2005-04-17 07:12:38 UTC
Well, that's not exactly as extended. But ok. It's in the same direction.
Comment 3 Thomas Bettler 2005-04-17 07:52:03 UTC
Yes, I suppose not earlier, because have a look how long the duplicate already has been fooling around... *I'm just realist