Speaker
Description
The rise of AI/ML in HPC applications is also driving the need for suitable storage abstractions such as the key-value (KV) stores. These abstractions pose new challenges for the HPC I/O stack. Enterprise KV stores are not well suited for HPC applications, and entail customization and cumbersome end-to-end KV design to extract the applications needs. To this end, I will present BESPOKV, an adaptive, extensible, and scale-out KV store framework. BESPOKV decouples the KV store design into the control plane for distributed management and the data plane for local data store. BESPOKV takes as input a single-server KV store, called a datalet, and transparently enables a scalable and fault-tolerant distributed KV store service. The resulting distributed stores are also adaptive to consistency or topology requirement changes and can be easily extended for new types of services. I’ll show that BESPOKV-enabled distributed KV stores scale horizontally to a large number of nodes, and performs comparably and sometimes better than the state-of-the-art systems.
