I have been experimenting trying to speed up long emerge sessions using openmosix. I have openmosix-sources-2.4.19-r5 and openmosix-user-0.2.4-r1. My cluster appears to come up fine. The home node is a k6 200, and the secondary node is an athlon 1700. I have configred MFS with dfsa=1. Heavily cpu bound test processes automatically migrate from the home node to the other node. However, compilation inside an ebuild does not. I had expected this to be something that 'just works'. Is this a bug, or just unrealistic expectations? I guessed that processes were not migrating because they are performing sufficient IO on their home node that openmosix thinks it would be a bad compromise to migrate them. changing PORTAGE_TMPDIR to something inside /mfs/2/ helped a little; gcc processes migrated briefly then came back home. However I also started getting intermittant compile failures. I am not sure whether this is an openmosix problem, or portage problem. One failure was repeatable - output attached.
Created attachment 4172 [details] output from a repeatable ebuild failure on openmosix MFS, which works fine without MFS
On my cluster compilation inside an ebuild (using emerge) migrates very well. Processes using extreme i/o like configure do not migrate. Be sure to set -j to at least 3 or 5 to give openmosix more time to migrate processes. Try using migrate like this: "migrate pid {OpenMosix-ID|home|balance}" to force openmosix to migrate your ebuild... (on my machines i don't need that... Athlon XP 1800+ and Amd 5x86-133 as a extreme cluster). Or use openmosixview to manipulate the openmosix-"speed" of your nodes to force migration earlier. Please try the things above without PORTAGE_TMPDIR. BTW: What kind of network are you using?
I had -j 6, and was using a pair of 100M 3COM 3c905 over a crossover cable. Without the PORTAGE_TMPDIR I had tried using 'migrate' to move the gcc (and other) processes. They migrate to the other node, but very quickly come back to their home. The speed ratings started out with the home node 7 times slower than the other machine. I had tried decreasing the home node to 10 times slower again, but no difference. If I was quick, and the gcc task slow, there was just about time to check /proc/pid/nmigs. This number was often above 20. Migrating wasnt a problem (as I would expect due the cpu imbalance) - they just dont stay migrated for very long. I think youve confirmed that the performance issues are not gentoo-specific. I will raise this question on the openmosix list. Re the compile failure when using MFS. The underlying filsystems are all reiser, if that makes a difference.
