Skip to content

Benchmarking Overview#

In each binary release of RonDB we also include a number of benchmark programs. This includes both scripts to execute the benchmarks as well as the required binaries to execute those benchmarks.

Currently this is only included in the Linux/x86 releases of RonDB, even though this is in the process of changing.

The benchmarks we focus on are the following:

  • Sysbench

    This is a simple benchmark that uses the MySQL C client to send simple SQL statements to RonDB. It is quite flexible and can be adapted to test many different things in RonDB.

  • YCSB (Yahoo Cloud Serving Benchmark)

    This is a benchmark that can use both the MySQL JDBC driver and it can also use ClusterJ. It contains six different workloads, mostly the operations are simple read or write operations on often a bit bigger rows. It is thus mainly a benchmark of key-value operations. We use it to show the LATS characteristics of RonDB (low Latency, high Availability, high Throughput and scalable Storage). It gives us the possibility to benchmark throughput while showing latency at the same time. We can show throughput and latency while restarting nodes. This gives a good view on our availablilty. Finally we can show performance with in-memory payload and we can show scalable Storage using disk columns for the payload data. This shows our throughput and latency also for disk-based data.

  • DBT2

    This is an open source variant of the TPC-C benchmark. It represents a slightly more complex application. It also uses the MySQL C client to issue SQL statements to RonDB. The SQL statements are using stored procedures.

  • DBT3

    This is an open source variant of the TPC-H benchmark. This benchmark executes complex queries with a lot of data in the database. It stresses the query execution engine in RonDB. RonDB supports pushing some complex join queries into the RonDB data nodes, thus achieving parallel query capabilities of RonDB. The main performance booster here is that the RonDB data nodes can perform parallel filtering of rows.

  • flexAsynch

    This is a simple benchmark focusing on testing RonDB as a key-value store. It uses the C++ NDB API to perform massive amounts of parallel read, insert, update and delete operations. It has been used to show how RonDB can handle massive amounts of key lookups.

  • ClusterJ benchmark

    ClusterJ is the native Java RonDB API. The benchmark focuses on performing simple key lookup operations towards RonDB. Makes it possible to show how batching can improve performance using ClusterJ and the C++ NDB API.

We will also discuss in this document how to mix various benchmarks. Given that RonDB is a distributed DBMS it is quite easy to execute more than one application towards it concurrently.

Currently we set up 4 different preconfigured RonDB benchmarks as part of managed RonDB. These are:

# Multiple MySQL Servers running Sysbench in parallel
/home/mysql/benchmarks/sysbench_multi
# Single MySQL Server running Sysbench
/home/mysql/benchmarks/sysbench_single
# Multiple MySQL Servers running DBT2 in parallel
/home/mysql/benchmarks/dbt2_multi
# Single MySQL Server running DBT2
/home/mysql/benchmarks/dbt2_single

The multi benchmarks use all MySQL Servers in the RonDB setup, single uses a single MySQL Server.

The description in this chapter assumes that you are using a cluster that has been started using managed RonDB. See the chapter on managed RonDB to see how this is done. Alternatively, you may have created a RonDB cluster locally with the rondb-docker repository, which uses the same benchmarking setup.

Mixing Benchmarks#

One of the requirements on RonDB is to be able to combine massive amounts of OLTP transactions with a set of complex query executions. Thus to execute a combination of Sysbench, YCSB, DBT2 and DBT3 is an interesting manner to see how RonDB can handle mixed workloads. In particular combining complex queries in DBT3 with other OLTP benchmarks is interesting.