Skip to main content

Memory Deduplication: An Effective Approach to Improve the Memory System

Deng, Yuhui, Huang, Xinyu, Song, Liangshan, Zhou, Yongtao, Wang, Frank Z. (2017) Memory Deduplication: An Effective Approach to Improve the Memory System. Journal of Information Science and Engineering, 33 (5). pp. 1103-1120. ISSN 1016-2364. (doi:10.6688/JISE.2017.33.5.1) (KAR id:69677)

This is the latest version of this item.

PDF Author's Accepted Manuscript
Language: English
Download (568kB) Preview
[thumbnail of MemoryDeduplication.pdf]
This file may not be suitable for users of assistive technology.
Request an accessible format
Official URL


Programs now have more aggressive demands of memory to hold their data than before. This paper analyzes the characteristics of memory data by using seven real memory traces. It observes that there are a large volume of memory pages with identical contents contained in the traces. Furthermore, the unique memory content accessed are much less than the unique memory address accessed. This is incurred by the traditional address-based cache replacement algorithms that replace memory pages by checking the addresses rather than the contents of those pages, thus resulting in many identical memory contents with different addresses stored in the memory. For example, in the same file system, opening two identical files stored in different directories, or opening two similar files that share a certain amount of contents in the same directory, will result in identical data blocks stored in the cache due to the traditional address-based cache replacement algorithms. Based on the observations, this paper evaluates memory compression and memory deduplication. As expected, memory deduplication greatly outperforms memory compression. For example, the best deduplication ratio is 4.6 times higher than the best compression ratio. The deduplication time and restore time are 121 times and 427 times faster than the compression time and decompression time, respectively. The experimental results in this paper should be able to offer useful insights for designing systems that require abundant memory to improve the system performance.

Item Type: Article
DOI/Identification number: 10.6688/JISE.2017.33.5.1
Additional information: The Best Paper Award of all papers published in the Journal of Information Science and Engineering in 2017
Uncontrolled keywords: Memory deduplication, address-based cache, content-based cache, memory compression, data characteristics
Subjects: Q Science > QA Mathematics (inc Computing science) > QA 75 Electronic computers. Computer science
Divisions: Divisions > Division of Computing, Engineering and Mathematical Sciences > School of Computing
Depositing User: Frank Wang
Date Deposited: 19 Oct 2018 04:23 UTC
Last Modified: 16 Feb 2021 13:58 UTC
Resource URI: (The current URI for this page, for reference purposes)
Wang, Frank Z.:

Available versions of this item

  • Depositors only (login required):


Downloads per month over past year