Similarity and Location Aware Scalable Deduplication System for Virtual Machine Storage Systems

Ongoing advancement in technology lead to ever-increasing storage capacities. Several techniques based on delta-encoding and duplicate block suppression have been shown to reduce storage over-heads, with varying requirements for resources such as computation and memory. Data deduplication has gained increasing attention and Ongoing Popularity as a space efficient approach in backup storage systems. Main challenges for centralized data deduplication is the capability of fingerprint-index search. We propose SiLo, a near-exact and salable deduplication system that professionally and complementary exploits similarity and locality of data streams to achieve high duplicate abstract�Data deduplication has gained increasing attention and popularity as a space-efficient approach in support elimination, throughput, and well balanced load at extremely low RAM overhead. The main scheme after SiLo is to representation and develop more similarity by grouping strongly correlated small files into a segment and segmenting large files, and to leverage the locality in the data stream by grouping contiguous segments into blocks to capture similar and duplicate data missed by the probabilistic similarity detection .By sensibly enhancing comparison through the development of locality and vice- verse. SiLo and employs a locality based stateless routing algorithm to paralleling and distribute data blocks to multiple backup nodes. SiLo is able to significantly reduce RAM usage for index look up, achieve the near-exact efficiency of duplicate elimination, a high deduplication throughput, and obtain load balance among backup nodes.