Bug 411173 – sys-cluster/orangefs-2.8.5 - a branch of the PVFS2 cluster filesystem

• Product: Gentoo Linux
• Status: UNCONFIRMED ---

[Description Andrew Savchenko 2012-04-07 17:47:01 UTC]

Hello,

I want to introduce formally new package (but actually it is not): OrangeFS — HPC-oriented distributed filesystem.

I set up a small HPC cluster (16 nodes, 16 hdds, 128 cores). I was in need for a distributed data storage to be used on our nodes. After some analysis of available solutions I chose OrangeFS (branch of PVFS2). Full discussion is available here:
http://forums.gentoo.org/viewtopic.php?p=7006816
The main conclusions and some test results are here:
http://forums.gentoo.org/viewtopic.php?p=7006816#6996204

In short, PVFS2 was once in portage (sys-cluster/pvfs2), but was removed due to problems with kernel module on recent kernels; so it's not that new. IMO this removal was premature, it was just days or weeks before updated version was released. In fact kernel module support is not strictly required for this filesystem to operate, see details below.

So I took the latest sys-cluster/pvfs2 ebuild with all patches and aux files as a ground zero. It was heavily reworked to accommodate all feature of orangefs, which are possible to support using current in-tree packages. Also I rewrote init scripts to add some cluster-friendly features, e.g. client startup may be optionally delayed until all server nodes will be ready to serve requests.

As for the kernel support, currently all kernels up to 3.2.x are support. 3.3.x is not, because it contains some heavy changes to the related infrastructure. OrangeFS developers are working on this, but this will take some considerable time.

But it is possible to use OrangeFS without kernel support at all:
1) FUSE client may be used instead of kernel-based one. Functionality is about the same. 
2) OrangeFS server is userspace and does not need kernel support at all. 
3) It is possible to use filesystem without VFS at all (yes, no mouts and usual file access). ROMIO API allows to access data directly from application using parallel I/O routines.

I'll upload all required files in a while, there are plenty of patches and this will take some time. For convenience the most recent version will be always available at my personal overlay "bircoph", which is registered at layman.

I can maintain this package if someone will proxy me. Perhaps the better idea will be to put it in the science overlay, because this distributed filesystem is HPC-targeted and fits the scope of the science overlay greatly; I have write access there, so I can do this, if there are no objections.

[Comment 1 Andrew Savchenko 2012-04-07 17:48:29 UTC]

Created attachment 308113 [details]
orangefs-2.8.5.ebuild

Comments and messages in the ebuild should be descriptive enough.

[Comment 2 Andrew Savchenko 2012-04-07 17:48:57 UTC]

Created attachment 308115 [details]
metadata.xml

[Comment 3 Andrew Savchenko 2012-04-07 17:50:20 UTC]

Created attachment 308117 [details, diff]
orangefs-2.8.5-fuse-install.patch

Fixes fuse install client path for Gentoo sandbox environment.

[Comment 4 Andrew Savchenko 2012-04-07 17:51:12 UTC]

Created attachment 308119 [details, diff]
orangefs-2.8.5-fuse-perms.patch

Upstream patch to fix FUSE permission change bug.

[Comment 5 Andrew Savchenko 2012-04-07 17:52:18 UTC]

Created attachment 308123 [details, diff]
orangefs-2.8.5-layout.patch

Upstream suggested change that adds ability to change default data distribution layout.

[Comment 6 Andrew Savchenko 2012-04-07 17:52:56 UTC]

Created attachment 308125 [details, diff]
orangefs-2.8.5-linux-3.1.patch

Upstream patch for 3.1 kernels only.

[Comment 7 Andrew Savchenko 2012-04-07 17:53:42 UTC]

Created attachment 308127 [details, diff]
orangefs-2.8.5-linux-3.2.patch

Upstream patch for 3.2 kernels only.

[Comment 8 Andrew Savchenko 2012-04-07 17:55:02 UTC]

Created attachment 308129 [details, diff]
orangefs-2.8.5-split-kernapps.patch

Gentoo-specific patch to split installation of kernel module and related utilities.

[Comment 9 Andrew Savchenko 2012-04-07 17:56:51 UTC]

Created attachment 308133 [details, diff]
orangefs-2.8.5-static-server.patch

Fixes build for static server binary, patch was accepted upstream.
(Static libraries and binaries are optional and USE controlled.)

[Comment 10 Andrew Savchenko 2012-04-07 17:59:14 UTC]

Created attachment 308137 [details, diff]
orangefs-2.8.5-vis.patch

This one allows to disable SDL automagick around subset of tools and to build them optionally. Currently VIS tools are disabled in the ebuild, because they are broken. This patch is accepted by upstream for testing.

[Comment 11 Andrew Savchenko 2012-04-07 17:59:56 UTC]

Created attachment 308139 [details]
pvfs2-client-conf.d

pvfs2-client conf.d file

[Comment 12 Andrew Savchenko 2012-04-07 18:00:26 UTC]

Created attachment 308141 [details]
pvfs2-client-init.d

pvfs2-client init.d script

[Comment 13 Andrew Savchenko 2012-04-07 18:01:25 UTC]

Created attachment 308143 [details]
pvfs2-server-conf.d

pvfs2-server conf.d file

[Comment 14 Andrew Savchenko 2012-04-07 18:01:57 UTC]

Created attachment 308145 [details]
pvfs2-server-init.d

pvfs2-server init.d script