Database Engines Architecture
The group has a long tradition of exploring the architecture of database engines, especially in the context of distributed data processing. Starting with our pioneer work on data replication (we were the first group to suggest to use agreement protocols to address the data replication problem) to engines eventually licensed to industry (Crescando, which was used by Amadeus for several years) and including involvement in industry projects such as Oracle's RAPID, the group has made and continues making many fundamental contributions to databases and their architecture.
NVM
Koutsoukos, Dimitrios Korenberg Friedman, Michal, Dr. Klimovic, Ana, Prof. Dr.
Persistent or Non Volatile Memory (PMEM) has recently become commercially available under several configurations with different purposes and goals. Despite the attention to the topic, there is no comprehensive empirical analysis of existing relational database engines under different PMEM modes. Such a study is essential to understand how database workloads and engines can benefit from the various hardware configurations. To this end, we analyze three different engines (PostgreSQL, MySQL, and SQLServer) under typical workloads (TPC-C and TPC-H) using PMEM as persistent memory in AppDirect mode and PMEM as volatile memory in Memory mode. Based on our findings, we provide insights based on how the different engines behave with PMEM and how various configurations and types of queries perform. Our results show that using PMEM in Memory mode does not offer any performance advantage despite the larger volatile memory capacity. Furthermore, using PMEM as persistent storage usually speeds up query execution, but with some caveats, the I/O path is not fully optimized. However, the additional performance benefit comes at a high cost, both in storage and CPU, since PMEM requires high-end processors. Therefore, to exploit PMEM in databases and tune relational engines to take advantage of this new technology in the best way possible, our study provides an important starting point by exploring a variety of database engines and parameters.
Parallel and Distributed Joins
external pageBarthels, Claude, Dr.call_made external pageBalkesen, Cagri, Dr.call_made external pageTeubnercall, Jens,Prof.call_made external pageÖzsu,Tamercall_made
Join processing in database systems is a complex and demanding operation. The architectural changes introduced with multi-core processors have triggered a redesign of main-memory join algorithms for relational database systems. Traditionally, there have been sorting and hashing-based approaches for implementing joins. However, in the last few years, several diverging views have appeared regarding designing and implementing main-memory joins on multi-core processors. In this project, we have investigated an extensive set of join approaches in terms of algorithms and implementations on a broad range of recent multi-core platforms. Our investigations have provided conclusive answers to the existing controversies in the literature.
RDMA and Data Processing
external pageBarthels, Claude, Dr.call_made Müller, Ingo, Dr.
Concurrency control is a cornerstone of distributed database engines and storage systems. In pursuit of scalability, a common assumption is that Two-Phase Locking (2PL) and Two-Phase Commit (2PC) are not viable solutions due to their communication overhead. Recent results, however, have hinted that 2PL and 2PC might not have such a bad performance. Nevertheless, there has been no attempt to measure how a state-of-the-art implementation of 2PL and 2PC would perform on modern hardware. This project aims to establish a baseline for concurrency control mechanisms on thousands of cores connected through a low-latency network. We developed a distributed lock table supporting all the standard locking modes used in database engines. We focus on solid consistency in the form of strict serializability implemented through strict 2PL but also explore read-committed and repeatable-read, two common isolation levels used in many systems. We do not leverage known optimizations in the locking or commit parts of the protocols. The surprising result is that, for TPC-C, 2PL and 2PC can be made to scale to thousands of cores and hundreds of machines, reaching a throughput of over 21 million transactions per second with 9.5 million New Order operations per second. Since most existing relational database engines use some form of locking for implementing concurrency control, our findings provide a path for such systems to scale without significantly redesigning transaction management. Our implementation relies on Remote Direct Memory Access (RDMA) to achieve these results. This technology is commonly available on Infiniband and Ethernet networks, making the results valid across many systems and platforms, including database appliances, data centers, and cloud environments.