Balancing Repair Bandwidth and Sub-Packetization in Erasure-Coded Storage via Elastic Transformation

Introduction

Erasure coding provides high fault-tolerant storage with significantly low redundancy overhead, at the expense of high repair bandwidth. While there exist access-optimal codes that theoretically minimize both the repair bandwidth and the amount of disk reads, they also incur a high sub-packetization level, thereby leading to non-sequential I/Os and degrading repair performance. We propose elastic transformation, a framework that transforms any base code into a new code with smaller repair bandwidth for all or a subset of nodes, such that it can be configured with a wide range of sub-packetization levels to limit the non-sequential I/O overhead. We prove the fault tolerance of elastic transformation and model numerically the repair performance with respect to a sub-packetization level. We further prototype and evaluate elastic transformation atop HDFS, and show how it reduces the single-block repair time of the base codes and access-optimal codes in a real network setting.

Publication

Download

License

The source code of the prototype is released under the Apache license 2.0 and is restricted for academic purposes only. Commercial use of the source code is not allowed.