Improvement in Data Deduplication Technique in Windows Environment Using File Metadata.

In computing, data deduplication is a specialized data compression technique for eliminating duplicate copies of repeating data. In this paper we use the data deduplication algorithm, in which unique parts of data, are identified and stored during a process of analysis. As the analysis continues, other parts… are compared to the stored copy and whenever a match occurs, the duplicate parts are replaced with a small reference that points to the stored parts of data. The same byte pattern in some part of complete file may occur thousands of times (the match frequency is dependent on the part (chunk) size), the amount of data that must be stored or transferred can be greatly reduced. Our techniques are provably accurate, yet run with very low memory requirements and avoid overheads associated with maintaining large deduplication tables. To remove the overhead associate with deduplication tables we examine the file metadata before calculating the hash values which are used to detect duplicate chunks in file.