Goals:

• Establish a comprehensive overview of common error scenarios and performance metrics relevant in the context of distributed database systems, including hardware malfunctions, software bugs, and security breaches.
• Develop a (single-node) database system that can act like a distributed database system, but allows configuration of metrics such as read/write latency distributions, throughput, and error scenarios including network latency, packet loss, server failures.
• Validate system correctness by quantifying the degree to which system behavior follows the configured parameters.
• Measure the precision of benchmarking results produced with a third-party database benchmarking framework such as YCSB.

Description:
This thesis introduces a novel approach to database benchmarking by simulating a wide range of error scenarios and performance metrics, offering a more nuanced view of system resilience and operational efficiency. By allowing users to configure aspects like read and write latency distributions, throughput, and error conditions, the framework aims to replicate the complex challenges faced by databases in real-world scenarios. Through this simulation, database administrators and developers can better understand how different systems respond to adverse conditions, leading to more informed decisions in database selection and configuration. This project seeks to bridge the gap between theoretical benchmarking and practical system performance, offering a tool that prepares databases for the unpredictability of real-world operations.

Further Reading:
[0] Paper: Who Watches the Watchmen? On the Lack of Validation in NoSQL Benchmarking (2015)
[1] Bachelor's Thesis: Simulation of Replication and Sharding With the SickStore Database System (2015)