NVHT: An efficient key-value storage library for non-volatile memory

作者:Huang, Kaixin; Zhou, Jie; Huang, Linpeng*; Shen, Yanyan
来源:Journal of Parallel and Distributed Computing, 2018, 120: 339-354.
DOI:10.1016/j.jpdc.2018.02.013

摘要

Non-Volatile Memory (NVM) promises persistence, byte-addressability and DRAM-like read/write latency. These properties indicate that NVM has the potential to be incorporated with key-value stores to achieve high performance and durability simultaneously. Specifically, data can be stored in NVM inherently without DRAM buffering, which eliminates expensive disk I/Os and data format transformation cost. However, several challenges such as data inconsistency and write endurance arise along with the benefits. We propose a library named NVHT to provide APIs for NVM-based key-value store operations. In NVHT, we introduce non-volatile pointer to solve the dynamic address mapping problem and design a wear-out-aware memory allocator for NVM. The core of NVHT is a novel NVM-friendly hash table structure. NVHT guarantees consistency using a log-based mechanism. The experimental results show that compared with LevelDB and BerkeleyDB running on a DRAM-based file system, NVHT achieves more than 2x and 4x speedup for insert and search operation respectively. Compared with in-memory key value store system Redis, NVHT still achieves higher transaction performance in terms of random update throughput (up to 1.5x and 2.5x for RDB scheme and AOF scheme, respectively).