As the demand of data reliability becomes more and more larger, most of today's storage systems adopt erasure codes to assure the data could be reconstructed when suffering from physical device failures. In order to fast recover the lost data from a single failure, recovery optimization methods have attracted a lot of attention in recent years. However, most of the existing optimization methods focus on homogeneous devices, ignoring the fact that the storage devices are usually heterogeneous. In this paper, we propose a new recovery optimization method named HSR (Heterogeneous Storage Recovery) method, which uses both loads and speed rate among physical devices as the optimization target, in order to further improve the recovery performance for heterogeneous devices. The experiment results show that, compared to existing popular recovery optimization methods, HSR method gains much higher recovery speed over heterogeneous storage devices.

With the rapid growth on data scale and complexity, single disk failure recovery becomes very important for erasure coded storage systems. In this paper, we propose a new strip-switched deployment method, which utilizes the feature that strips of each stripe of erasure codes could be switched, and uses simulated annealing algorithm to search for the proper strip-deployment on the stack level to balance the read accesses, in order to improve the recovery performance. The analysis and experiments results show that SSDM could effectively improve the single failure recovery performance.

Erasure codes have been considered as one of the most promising techniques for data reliability enhancement and storage efficiency in modern distributed storage systems. However, erasure codes often suffer from a time-consuming coding process which makes them nearly impractical. The opportunity to solve this problem probably rely on the parallelization of erasure-code-based application on the modern multi-/many-core processors to fully take advantage of the adequate hardware resources on those platforms. However, the complicated data allocation and limited I/O throughput pose a great challenge on the parallelization. To address this challenge, we propose a general multi-threaded parallel coding approach in this work. The approach consists of a general multi-threaded parallel coding model named as MTPerasure, and two detailed parallel coding algorithms, named as sdaParallel and ddaParallel, respectively, adapting to different I/O circumstances. MTPerasure is a general parallel coding model focusing on the high level data allocation, and it is applicable for all erasure codes and can be implemented without any modifications of the low level coding algorithms. The sdaParallel divides the data into several parts and the data parts are allocated to different threads statically in order to eliminate synchronization latency among multiple threads, which improves the parallel coding performance under the dummy I/O mode. The ddaParallel employs two threads to execute the I/O reading and writing on the basis of small pieces independently, which increases the I/O throughput. Furthermore, the data pieces are assigned to the coding thread dynamically. A special thread scheduling algorithm is also proposed to reduce thread migration latency. To evaluate our proposal, we parallelize the popular open source library jerasure based on our approach. And a detailed performance comparison with the original sequential coding program indicates that the proposed parallel approach outperforms the original sequential program by an extraordinary speedups from 1.4x up to 7x, and achieves better utilization of the computation and I/O resources.

Locally repairable codes (LRCs) have attracted much interest recently due to their applications in distributed storage systems. In an [n,k,d] linear code, a code symbol is said to have locality r if it can be repaired by accessing at most r other code symbols. An (n,k,r) LRC with locality r for the information symbols has minimum distance d≤n-k-⌈k/r⌉+2. In this letter, we study single-parity LRCs where every repair group contains exactly one parity symbol. Firstly, we give a new characterization of single-parity LRCs based on the standard form of generator matrices. For the optimal single-parity LRCs meeting the Singleton-like bound, we give necessary conditions on the structures of generator matrices. Then we construct all the optimal binary single-parity LRCs meeting the Singleton-like bound d≤n-k-⌈k/r⌉+2.

RAID has been widely deployed in disk array storage systems to manage both performance and reliability simultaneously. RAID conducts two performance-critical operations during disk failures known as degraded reads/writes and recovery process. Before the recovery process is complete, reads and writes are degraded because data is reconstructed using data redundancy. The performance of degraded reads/writes is critical in order to meet stipulations in customer service level agreements (SLAs), and the recovery process affects the reliability of a storage system considerably. Both operations require fast data reconstruction. Among the erasure codes for fast reconstruction, Local Reconstruction Codes (LRC) are known to offer the best (or optimal) trade-off between storage overhead, fault tolerance, and the number of disks involved in reconstruction. Originally, LRC was designed for fast reconstruction in distributed cloud storage systems, in which network traffic is a major bottleneck during reconstruction. Thus, LRC focuses on reducing the number of disks involved in data reconstruction, which reduces network traffic. However, we observe that when LRC is applied to primary array storage systems, a major bottleneck in reconstruction results from uneven disk utilization. In other words, underutilized disks can no longer receive I/O requests as a result of the bottleneck of overloaded disks. Uneven disk utilization in LRC is due to its dedicated group partitioning policy to achieve the Maximally Recoverable property. In this paper, we present Distributed Reconstruction Codes (DRC) that support fast reconstruction in primary array storage systems. DRC is designed with group shuffling policy to solve the problem of uneven disk utilization. Experiments on real-world workloads show that DRC using global parity rotation (DRC-G) improves degraded performance by as much as 72% compared to RAID-6 and by as much as 35% compared to LRC under the same reliability. In addition, our study shows that DRC-G reduces the recovery process completion time by as much as 52% compared to LRC.

With the increase of data quantity, people have begun to attach importance to cloud storage. However, numerous security accidents occurred to cloud servers recently, thus triggering thought about the security of traditional single cloud. In other words, traditional single cloud can't ensure the privacy of users' data to a certain extent. To solve those security issues, multi-cloud systems which spread data over multiple cloud storage servers emerged. They employ a series of erasure codes and other keyless dispersal algorithms to achieve high-level security. But non-systematic codes like RS require relatively complex arithmetic, and systematic codes have relatively weaker security. In terms of keyless dispersal algorithms, they avoid key management issues but not suit to complete parallel optimization or deduplication which is important to limited cloud storage resources. So in this paper, we design a new kind of XOR-based non-systematic erasure codes - Privacy Protecting Codes (PPC) and a SIMD encoding algorithm for better performance. To achieve higher-level security, we put forward a novel deduplication-friendly dispersal algorithm called Hash Cyclic Encryption-PPC (HCE-PPC) which can achieve complete parallelization. With these new technologies, we present a multi-cloud storage system called CloudS. For better user experience and the tradeoffs between security and performance, CloudS provides multiple levels of security by various combinations of compression, encryption and coding schemes. We implement CloudS as a web application which doesn't require users to perform complicated operations on local.

Recent studies on switching fabrics mainly focus on the switching schedule algorithms, which aim at improving the throughput (a key performance metric). However, the delay (another key performance metric) of switching fabrics cannot be well guaranteed. A good switching fabric should be endowed with the properties of high throughput, delay guarantee, low component complexity and high-speed multicast, which are difficult for conventional switching fabrics to achieve. This has fueled great interest in designing a new switching fabric that can support large-scale extension and high-speed multicast. Motivated by this, we reuse the self-routing Boolean concentrator network and embed a model of multicast packet copy separation in front to construct a load-balanced multicast switching fabric (LB-MSF) with delay guarantee. The first phase of LB-MSF is responsible for balancing the incoming traffic into uniform cells while the second phase is in charge of self-routing the cells to their final destinations. In order to improve the throughput, LB-MSF is combined with the merits of erasure codes against packet loss. Experiments and analyses verify that the proposed fabric is able to achieve high-speed multicast switching and suitable for building super large-scale switching fabric in Next Generation Network(NGN) with all the advantages mentioned above. Furthermore, a prototype of the proposed switch is developed on FPGA, and presents excellent performance.

Rateless erasure code is an error correction code that is able to encode a message of k uncoded symbols into an infinite number of coded symbols. One may reconstruct the original message from any k(1+ε) coded symbols, where ε denotes the decoding inefficiency. This paper proposes a hybrid code that combines the stepping code and random code and name it as Stepping-Random (SR) code. The Part I (first k) coded symbols of SR code are generated with stepping code. The rest of the coded symbols are generated with random code and denoted as Part II coded symbols. The numerical results show that the new hybrid code is able to achieve a complete decoding with no extra coded symbol (ε=0) if all the Part I coded symbols are received without loss. However, if only a portion of Part I coded symbols are received, a high probability of complete decoding is still achievable with k+10 coded symbols from the combination of Part I and II. SR code has a decoding complexity of O(k) in the former and O((βk)3) in the latter, where β ∈ R for 0 ≤ β ≤ 1, is the fraction of uncoded symbols that fails to be reconstructed from Part I coded symbols.

For the efficient multicast distribution services on the Internet, suppressing the influence of packet loss is important issues. As a solution of this problem, Forward Error Correction (FEC) based on Reed-Solomon codes is usually used. However, in the case of content delivery services for a large amount of data, this approach is not suitable. In this paper, we focus on the erasure codes which are new approach of FEC and propose the efficient multicast video distribution method which combines the multicast distribution using erasure codes and direct request to the server. We implement proposal method and confirm its efficiency from the viewpoints of redundancy and processing time.