Database engines must utilize and adapt to the underlying hardware. New hardware such as non-volatile memory (NVM) and massively parallel processors, is changing fundamentally how database engines should be designed for better scalability, performance, and more functionality. This talk will examine recent hardware trends, challenges brought by such hardware, and propose solutions that judiciously utilize NVM, large DRAM, as well as the unprecedented high parallelism given by modern and future processors. The first part of the talk will show the enabling role of NVM in easing the centralized logging bottleneck that prevents us from unleashing the potential of high parallelism. We then turn to the functionality perspective: with the logging bottleneck removed, main-memory databases are able to scale up, but only so for the “right” workloads. We set out to devise concurrency control mechanisms that not only run fast, but also robustly against a wide spectrum of workloads. Finally, we take a tour in the world of synchronization, to see how synchronization primitives and database engines impact each other.
See more on this video at www.microsoft.com/en-us/research/video/fast-robust-transaction-processing-emerging-hardware/