Resource-efficient, performant in-memory KV stores for at-scale data centers
OA Version
Citation
Abstract
In-memory Key-Value (KV) stores are non-persistent storage backbones of an ever-growing number of services in data centers. They reduce latency and increase throughput by caching objects obtained from slow databases or storage systems. A single caching server can replace tens of backend database servers by absorbing massive database request loads. Unfortunately, current solutions are general-purpose and not fully aware of the characteristics and requirements of in-memory KV stores, which can lead to low responsiveness and resource efficiency.
This Ph.D. thesis begins by characterizing in-memory KV store workloads in contemporary data centers. Based on these characteristics, the thesis proposes several trait-aware systems designed to enhance the performance and resource efficiency of in-memory KV stores. At the operating system (OS) level, these systems target latency and utilization, while at the application level, they target power consumption and memory efficiency.
First, the thesis presents several CPU schedulers for hypervisors and operating systems that reduce the tail latency of in-memory KV stores. These schedulers do not treat all tasks equally. They dynamically identify I/O-bound tasks responsible for handling in-memory KV stores and shorten their CPU access latencies to enhance responsiveness.
Next, the thesis introduces Akita, a hypervisor CPU scheduler that improves the CPU utilization of cloud data centers. Unlike existing solutions, Akita allows critical and non-critical tasks to co-exist while maintaining high performance for critical ones, thus improving utilization. Akita slows down or ignores non-critical tasks when necessary to ensure high performance for critical ones.
Next, the thesis presents Peafowl, an application-level system that reduces power consumption of in-memory KV stores. Peafowl introduces the concept of in-application CPU scheduling to gain a deeper understanding of the workload. This stands in contrast to general-purpose OS schedulers, which are oblivious to workload characteristics. Peafowl uses in-application knowledge to learn the workload traits, such as off-peak periods, in order to pack the workload onto fewer CPU cores during low utilization periods, thereby saving power.
Finally, the thesis presents Caterpillar, an in-application memory manager that enhances the memory efficiency of KV stores with values that grow at various rates for different keys. Unlike existing KV stores, Caterpillar does not rely on oblivious OS for memory offloading. Instead, it features a low-overhead memory offloader that learns the workload's characteristics over time and categorizes it into smaller pieces suitable for keeping in memory, transferring to disks, or deleting.
Experiments with prototypes of these designs demonstrate their effectiveness. As in-memory KV stores are widely adopted, performant and resource-efficient KV stores can significantly enhance data centers' service quality, reduce costs, and, more importantly, mitigate environmental impacts.
Description
License
Attribution 4.0 International