Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 411173 - sys-cluster/orangefs-2.8.5 - a branch of the PVFS2 cluster filesystem
Summary: sys-cluster/orangefs-2.8.5 - a branch of the PVFS2 cluster filesystem
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: New packages (show other bugs)
Hardware: All Linux
: Normal enhancement (vote)
Assignee: Default Assignee for New Packages
Depends on:
Reported: 2012-04-07 17:47 UTC by Andrew Savchenko
Modified: 2021-03-27 03:22 UTC (History)
5 users (show)

See Also:
Package list:
Runtime testing required: ---

orangefs-2.8.5.ebuild (orangefs-2.8.5.ebuild,8.99 KB, text/plain)
2012-04-07 17:48 UTC, Andrew Savchenko
metadata.xml (metadata.xml,1.34 KB, text/plain)
2012-04-07 17:48 UTC, Andrew Savchenko
orangefs-2.8.5-fuse-install.patch (orangefs-2.8.5-fuse-install.patch,311 bytes, patch)
2012-04-07 17:50 UTC, Andrew Savchenko
Details | Diff
orangefs-2.8.5-fuse-perms.patch (orangefs-2.8.5-fuse-perms.patch,659 bytes, patch)
2012-04-07 17:51 UTC, Andrew Savchenko
Details | Diff
orangefs-2.8.5-layout.patch (orangefs-2.8.5-layout.patch,4.09 KB, patch)
2012-04-07 17:52 UTC, Andrew Savchenko
Details | Diff
orangefs-2.8.5-linux-3.1.patch (orangefs-2.8.5-linux-3.1.patch,3.92 KB, patch)
2012-04-07 17:52 UTC, Andrew Savchenko
Details | Diff
orangefs-2.8.5-linux-3.2.patch (orangefs-2.8.5-linux-3.2.patch,5.96 KB, patch)
2012-04-07 17:53 UTC, Andrew Savchenko
Details | Diff
orangefs-2.8.5-split-kernapps.patch (orangefs-2.8.5-split-kernapps.patch,571 bytes, patch)
2012-04-07 17:55 UTC, Andrew Savchenko
Details | Diff
orangefs-2.8.5-static-server.patch (orangefs-2.8.5-static-server.patch,625 bytes, patch)
2012-04-07 17:56 UTC, Andrew Savchenko
Details | Diff
orangefs-2.8.5-vis.patch (orangefs-2.8.5-vis.patch,2.35 KB, patch)
2012-04-07 17:59 UTC, Andrew Savchenko
Details | Diff
pvfs2-client-conf.d (pvfs2-client-conf.d,1.12 KB, text/plain)
2012-04-07 17:59 UTC, Andrew Savchenko
pvfs2-client-init.d (pvfs2-client-init.d,4.28 KB, text/plain)
2012-04-07 18:00 UTC, Andrew Savchenko
pvfs2-server-conf.d (pvfs2-server-conf.d,846 bytes, text/plain)
2012-04-07 18:01 UTC, Andrew Savchenko
pvfs2-server-init.d (pvfs2-server-init.d,4.38 KB, text/plain)
2012-04-07 18:01 UTC, Andrew Savchenko

Note You need to log in before you can comment on or make changes to this bug.
Description Andrew Savchenko gentoo-dev 2012-04-07 17:47:01 UTC

I want to introduce formally new package (but actually it is not): OrangeFS — HPC-oriented distributed filesystem.

I set up a small HPC cluster (16 nodes, 16 hdds, 128 cores). I was in need for a distributed data storage to be used on our nodes. After some analysis of available solutions I chose OrangeFS (branch of PVFS2). Full discussion is available here:
The main conclusions and some test results are here:

In short, PVFS2 was once in portage (sys-cluster/pvfs2), but was removed due to problems with kernel module on recent kernels; so it's not that new. IMO this removal was premature, it was just days or weeks before updated version was released. In fact kernel module support is not strictly required for this filesystem to operate, see details below.

So I took the latest sys-cluster/pvfs2 ebuild with all patches and aux files as a ground zero. It was heavily reworked to accommodate all feature of orangefs, which are possible to support using current in-tree packages. Also I rewrote init scripts to add some cluster-friendly features, e.g. client startup may be optionally delayed until all server nodes will be ready to serve requests.

As for the kernel support, currently all kernels up to 3.2.x are support. 3.3.x is not, because it contains some heavy changes to the related infrastructure. OrangeFS developers are working on this, but this will take some considerable time.

But it is possible to use OrangeFS without kernel support at all:
1) FUSE client may be used instead of kernel-based one. Functionality is about the same. 
2) OrangeFS server is userspace and does not need kernel support at all. 
3) It is possible to use filesystem without VFS at all (yes, no mouts and usual file access). ROMIO API allows to access data directly from application using parallel I/O routines.

I'll upload all required files in a while, there are plenty of patches and this will take some time. For convenience the most recent version will be always available at my personal overlay "bircoph", which is registered at layman.

I can maintain this package if someone will proxy me. Perhaps the better idea will be to put it in the science overlay, because this distributed filesystem is HPC-targeted and fits the scope of the science overlay greatly; I have write access there, so I can do this, if there are no objections.
Comment 1 Andrew Savchenko gentoo-dev 2012-04-07 17:48:29 UTC
Created attachment 308113 [details]

Comments and messages in the ebuild should be descriptive enough.
Comment 2 Andrew Savchenko gentoo-dev 2012-04-07 17:48:57 UTC
Created attachment 308115 [details]
Comment 3 Andrew Savchenko gentoo-dev 2012-04-07 17:50:20 UTC
Created attachment 308117 [details, diff]

Fixes fuse install client path for Gentoo sandbox environment.
Comment 4 Andrew Savchenko gentoo-dev 2012-04-07 17:51:12 UTC
Created attachment 308119 [details, diff]

Upstream patch to fix FUSE permission change bug.
Comment 5 Andrew Savchenko gentoo-dev 2012-04-07 17:52:18 UTC
Created attachment 308123 [details, diff]

Upstream suggested change that adds ability to change default data distribution layout.
Comment 6 Andrew Savchenko gentoo-dev 2012-04-07 17:52:56 UTC
Created attachment 308125 [details, diff]

Upstream patch for 3.1 kernels only.
Comment 7 Andrew Savchenko gentoo-dev 2012-04-07 17:53:42 UTC
Created attachment 308127 [details, diff]

Upstream patch for 3.2 kernels only.
Comment 8 Andrew Savchenko gentoo-dev 2012-04-07 17:55:02 UTC
Created attachment 308129 [details, diff]

Gentoo-specific patch to split installation of kernel module and related utilities.
Comment 9 Andrew Savchenko gentoo-dev 2012-04-07 17:56:51 UTC
Created attachment 308133 [details, diff]

Fixes build for static server binary, patch was accepted upstream.
(Static libraries and binaries are optional and USE controlled.)
Comment 10 Andrew Savchenko gentoo-dev 2012-04-07 17:59:14 UTC
Created attachment 308137 [details, diff]

This one allows to disable SDL automagick around subset of tools and to build them optionally. Currently VIS tools are disabled in the ebuild, because they are broken. This patch is accepted by upstream for testing.
Comment 11 Andrew Savchenko gentoo-dev 2012-04-07 17:59:56 UTC
Created attachment 308139 [details]

pvfs2-client conf.d file
Comment 12 Andrew Savchenko gentoo-dev 2012-04-07 18:00:26 UTC
Created attachment 308141 [details]

pvfs2-client init.d script
Comment 13 Andrew Savchenko gentoo-dev 2012-04-07 18:01:25 UTC
Created attachment 308143 [details]

pvfs2-server conf.d file
Comment 14 Andrew Savchenko gentoo-dev 2012-04-07 18:01:57 UTC
Created attachment 308145 [details]

pvfs2-server init.d script