New Research by Zhou Improves Data Deduplication in Backup Storage Systems

As TSYS School computer science professor Yi Zhou and his colleagues from Jinan University and the University of Maine explain in their new study, data deduplication has been widely used in backup storage systems to improve storage utilization and extend device lifetime by reducing data writes.  Inline deduplication, in particular, removes redundant data in real-time as data is being sent to the storage system.  However, it causes data fragmentation, meaning that logically consecutive chunks are physically scattered across various containers after data deduplication.  Many existing rewrite algorithms aim to alleviate the performance degradation due to fragmentation by rewriting fragmented duplicate chunks as unique chunks into new containers.  Unfortunately, these algorithms determine whether a chunk is fragmented based on a simple pre-set fixed value, ignoring the variance of data characteristics between data segments.  This means that when backups are restored, they often fail to select an appropriate set of old containers for rewrite, generating a substantial number of invalid chunks in retrieved containers.  To address this issue, Zhou and his colleagues propose an inline deduplication approach for storage systems, called InDe, which detects valid container utilization and dynamically adjusts the number of old container references in each segment.  InDe is designed to improve restore performance while maintaining high backup performance by identifying appropriate containers for the rewrite.  Lastly, Zhou and his colleagues are able to evaluate InDe using three real-world backup workloads.  The experimental results presented in the study, which appears in the current issue of ACM Transactions on Storage, show that compared to two other state-of-the-art algorithms, InDe improves restore speed by 1.3 to 2.4 times while also achieving similar backup performance.