NCCloud (formerly known as CloudNCFS) is a proof-of-concept prototype of a network-coding-based file system that aims at providing fault tolerance and reducing data repair cost when storing files using multiple-cloud storage (or any other kinds of raw storage devices). NCCloud is a proxy-based file system that interconnects multiple (cloud) storage nodes. It can be mounted as a directory on Linux, and file uploading/downloading are done by copying files to/from the mounted directory. NCCloud is built on FUSE, an open-source, programmable user-space file system that provides application programmable interfaces (APIs) for file system operations. From the point of view of user applications, NCCloud presents a file system layer that transparently stripes data across storage nodes.
Network codes for storage repair require that storage nodes encode the stored data during the repair process. However, this may not be feasible for some storage systems where nodes only provide the basic I/O functionalities but do not have the encoding capability. Our work is to adapt the benefits of network codes in the storage repair of a practical storage setting, by relaxing the encoding requirement of storage nodes.
NCCloud supports a variety of coding schemes, in particular the Functional Minimum Storage Regenerating (F-MSR) codes. Compared to traditional optimal erasure codes (e.g., Reed-Solomon), FMSR codes maintains the same storage overhead under the same data redundancy level, but uses less repair traffic during the recovery of a single failed storage node. NCCloud realizes regenerating codes in a practical cloud storage system that does not require any encoding/decoding intelligence on the cloud storage nodes.
NCCloud is built on another sister project NCFS.
$ tar xf nccloud-c++-1.0.tar.gz && cd nccloud-c++-1.0/nccloud
$ make
$ export LD_LIBRARY_PATH="../libfmsr/lib:../Jerasure/lib:$LD_LIBRARY_PATH" # or install the libraries any way you want
$ cp template/config_local config
$ mkdir -p store/{tmp,node{0..4}}
$ bin/nccloud config encode README template/config_local
$ bin/nccloud config repair 0 README config_local
$ bin/nccloud config decode README config_local
$ diff {,store/tmp/}README
$ diff {template,store/tmp}/config_local
$ bin/list_repo config
$ bin/nccloud config delete README config_local
$ bin/list_repo config
$ tar xf libfmsr-0.1.tar.gz && cd libfmsr-0.1 $ make && make test
NCCloud uses the Jerasure library for coding implementation.
The work is supported by grants from the University Grants Committee of Hong Kong (AoE/E-02/08 and ECS CUHK419212) and seed grants from the CUHK MoE-Microsoft Key Laboratory of Human-centric Computing and Interface Technologies.
NCCloud is developed by Advanced Network and System Research Laboratory in the Department of Computer Science and Engineering at the Chinese University of Hong Kong (CUHK). This project is also affiliated with the Institute of Network Coding at CUHK
Please contact Patrick P. C. Lee if you have any questions.