Erasure codes have been considered as one of the most promising techniques for data reliability enhancement and storage efficiency in modern distributed storage systems. However, erasure codes often suffer from a time-consuming coding process which makes them nearly impractical. The opportunity to solve this problem probably rely on the parallelization of erasure-code-based application on the modern multi-/many-core processors to fully take advantage of the adequate hardware resources on those platforms. However, the complicated data allocation and limited I/O throughput pose a great challenge on the parallelization. To address this challenge, we propose a general multi-threaded parallel coding approach in this work. The approach consists of a general multi-threaded parallel coding model named as MTPerasure, and two detailed parallel coding algorithms, named as sdaParallel and ddaParallel, respectively, adapting to different I/O circumstances. MTPerasure is a general parallel coding model focusing on the high level data allocation, and it is applicable for all erasure codes and can be implemented without any modifications of the low level coding algorithms. The sdaParallel divides the data into several parts and the data parts are allocated to different threads statically in order to eliminate synchronization latency among multiple threads, which improves the parallel coding performance under the dummy I/O mode. The ddaParallel employs two threads to execute the I/O reading and writing on the basis of small pieces independently, which increases the I/O throughput. Furthermore, the data pieces are assigned to the coding thread dynamically. A special thread scheduling algorithm is also proposed to reduce thread migration latency. To evaluate our proposal, we parallelize the popular open source library jerasure based on our approach. And a detailed performance comparison with the original sequential coding program indicates that the proposed parallel approach outperforms the original sequential program by an extraordinary speedups from 1.4x up to 7x, and achieves better utilization of the computation and I/O resources.

Locally repairable codes (LRCs) have attracted much interest recently due to their applications in distributed storage systems. In an [n,k,d] linear code, a code symbol is said to have locality r if it can be repaired by accessing at most r other code symbols. An (n,k,r) LRC with locality r for the information symbols has minimum distance d≤n-k-⌈k/r⌉+2. In this letter, we study single-parity LRCs where every repair group contains exactly one parity symbol. Firstly, we give a new characterization of single-parity LRCs based on the standard form of generator matrices. For the optimal single-parity LRCs meeting the Singleton-like bound, we give necessary conditions on the structures of generator matrices. Then we construct all the optimal binary single-parity LRCs meeting the Singleton-like bound d≤n-k-⌈k/r⌉+2.

RAID has been widely deployed in disk array storage systems to manage both performance and reliability simultaneously. RAID conducts two performance-critical operations during disk failures known as degraded reads/writes and recovery process. Before the recovery process is complete, reads and writes are degraded because data is reconstructed using data redundancy. The performance of degraded reads/writes is critical in order to meet stipulations in customer service level agreements (SLAs), and the recovery process affects the reliability of a storage system considerably. Both operations require fast data reconstruction. Among the erasure codes for fast reconstruction, Local Reconstruction Codes (LRC) are known to offer the best (or optimal) trade-off between storage overhead, fault tolerance, and the number of disks involved in reconstruction. Originally, LRC was designed for fast reconstruction in distributed cloud storage systems, in which network traffic is a major bottleneck during reconstruction. Thus, LRC focuses on reducing the number of disks involved in data reconstruction, which reduces network traffic. However, we observe that when LRC is applied to primary array storage systems, a major bottleneck in reconstruction results from uneven disk utilization. In other words, underutilized disks can no longer receive I/O requests as a result of the bottleneck of overloaded disks. Uneven disk utilization in LRC is due to its dedicated group partitioning policy to achieve the Maximally Recoverable property. In this paper, we present Distributed Reconstruction Codes (DRC) that support fast reconstruction in primary array storage systems. DRC is designed with group shuffling policy to solve the problem of uneven disk utilization. Experiments on real-world workloads show that DRC using global parity rotation (DRC-G) improves degraded performance by as much as 72% compared to RAID-6 and by as much as 35% compared to LRC under the same reliability. In addition, our study shows that DRC-G reduces the recovery process completion time by as much as 52% compared to LRC.

Recent studies on switching fabrics mainly focus on the switching schedule algorithms, which aim at improving the throughput (a key performance metric). However, the delay (another key performance metric) of switching fabrics cannot be well guaranteed. A good switching fabric should be endowed with the properties of high throughput, delay guarantee, low component complexity and high-speed multicast, which are difficult for conventional switching fabrics to achieve. This has fueled great interest in designing a new switching fabric that can support large-scale extension and high-speed multicast. Motivated by this, we reuse the self-routing Boolean concentrator network and embed a model of multicast packet copy separation in front to construct a load-balanced multicast switching fabric (LB-MSF) with delay guarantee. The first phase of LB-MSF is responsible for balancing the incoming traffic into uniform cells while the second phase is in charge of self-routing the cells to their final destinations. In order to improve the throughput, LB-MSF is combined with the merits of erasure codes against packet loss. Experiments and analyses verify that the proposed fabric is able to achieve high-speed multicast switching and suitable for building super large-scale switching fabric in Next Generation Network(NGN) with all the advantages mentioned above. Furthermore, a prototype of the proposed switch is developed on FPGA, and presents excellent performance.

For the efficient multicast distribution services on the Internet, suppressing the influence of packet loss is important issues. As a solution of this problem, Forward Error Correction (FEC) based on Reed-Solomon codes is usually used. However, in the case of content delivery services for a large amount of data, this approach is not suitable. In this paper, we focus on the erasure codes which are new approach of FEC and propose the efficient multicast video distribution method which combines the multicast distribution using erasure codes and direct request to the server. We implement proposal method and confirm its efficiency from the viewpoints of redundancy and processing time.