Skip to content

RonDB Tools#

There is a new GitHub tree. This tree is created by the RonDB team for use for experimentation with RonDB, running RonDB benchmarks, testing RonDB upgrades and system testing of RonDB. Its first use case was to setting up a cluster to perform more than 100M key lookups per second using RonDB.

The current set of tools uses Terraform to setup a set of VMs in AWS as directed by the Python configuration file config.py. Next it deploys the RonDB software and the required libraries on the RonDB nodes. It also deploys the benchmark tool Locust and the valkey-benchmark on the VMs used as benchmark clients.

The interactions with the tool goes through the script cluster_ctl.

The intention is to continue developing this tool, now it installs on VMs, next phase will install RonDB using RonDB Helm charts. Currently it is only targeting cluster setups in AWS. This is likely to be followed by other clouds as well.

How to deploy a RonDB cluster#

Installations required to work with RonDB tool#

At first it is necessary to install the following tools, Terraform, AWS CLI, python3, rsync and tmux if you want to use a lot of terminal windows.

Installing terraform is explained on this link.

On Mac OS X one best uses brew to install the other tools using the following command:

brew install awscli python3 rsync tmux

On Debian/Ubuntu the following commands can be used:

sudo apt-get update && sudo apt-get install -y awscli python3 rsync tmux

In order to run this tool you need an AWS API key to create cloud resources. Run the following command to configure the AWS access key id and the access key.

aws configure

With this preparatory step you are now prepared to create a RonDB cluster. Creating a RonDB happens in 4 phases.

  • Edit the configuration in config.py

  • Run ./cluster_ctl terraform

  • Run ./cluster_ctl install

  • Run ./cluster_ctl start

The terraform, install and start can be combined into one command ./cluster_ctl deploy.

After this command you have created the VMs, installed all the necessary binaries and libraries required to run the various VMs in the RonDB cluster and finally also started the RonDB cluster.

At this point we have also installed Locust software on the benchmark clients as well as valkey-benchmark. To run the Locust benchmark two more steps are required.

  • Run ./cluster_ctl populate

  • Run ./cluster_ctl start_locust

These two steps creates a table, populates the table with data and starts all the Locust workers in the benchmark clients. These two commands can be combined into one command ./cluster_ctl bench_locust.

The start command also started monitoring of the RonDB cluster. The monitoring happens in a VM that runs Grafana and this VM is supported by another VM that runs Prometheus. The start command will return an https address from where you can see the RonDB dashboards.

The start_locust command will return a https address to a web page from where you can start and stop the Locust benchmark, it will also show the througput and latency of the operations it performs.

Command reference for the RonDB tool#

Some sub-commands take a NODES argument. It is a comma-separated list of node names and node types. If given, the command will operate only on matching nodes, otherwise on all nodes. Available node types are:

  • ndb_mgmd

  • ndbmtd

  • mysqld

  • rdrs

  • prometheus

  • grafana

  • bench

terraform#

./cluster_ctl terraform

Configure, initialize and apply terraform as needed. Performs the following steps:

  • create or update terraform.tfvars

  • call terraform init unless it is already initialized

  • call terraform apply

install#

./cluster_ctl install [NODES]

Install necessary software and configuration files. Depending on the node type this may include RonDB, prometheus, grafana and locust.

start#

./cluster_ctl start [NODES]

Start or restart RonDB services. This doesn’t include benchmark services.

Deploy#

./cluster_ctl deploy [NODES]

This command is a convenience command equivalent to terraform, install and start.

stop#

./cluster_ctl stop [NODES]

Stop RonDB services. This doesn’t include benchmark services.

populate#

./cluster_ctl populate [--cols COLUMNS] [--rows ROWS] [--types COLUMN_TYPES]

Creates and populates a table for use by the Locust benchmark.

The default for --cols is 10. The default for --rows is 100000. The column types supported in --types is INT, VARCHAR and INT_AND_VARCHAR. INT_AND_VARCHAR is the default.

start_locust#

./cluster_ctl start_locust [--rows ROWS] [--workers WORKERS]

Start or restart locust services. You need to run populate first.

--rows must be set to the same value as in populate. --workers is by default equal to the number of CPUs in the benchmark client VMs minus one (one CPU is used by Locust for the manager part of the Locust services, the remaining are one CPU per Locust worker.

bench_locust#

./cluster_ctl bench_locust [--cols COLUMNS] [--types COLUMN_TYPES] [--rows ROWS] [--workers WORKERS]

This is a convenience command that first calls populate and next calls start_locust.

stop_locust#

./cluster_ctl stop_locust

Stops Locust

list#

./cluster_ctl list

Print node information, including node name, node type, public and private IP, NDB Node IDs, and running processes.

open_tmux#

./cluster_ctl open_tmux [NODES]

Create a tmux session with ssh connections opened to all applicable nodes.

When attached to the tmux session, you can press Ctrl-b n to go to the next window, and Ctrl-b d to detach.

If a session already exists, ignore the NODES argument and reattach to it.

To connect to a different set of nodes, use kill_tmux first.

See also the ssh command.

kill_tmux#

./cluster_ctl kill_tmux

Kill the tmux session created by open_tmux.

ssh#

./cluster_ctl ssh NODE_NAME [COMMAND ...]

Connect to a node using SSH. Run the list command to discover the node names available.

cleanup#

./cluster_ctl cleanup

This command cleans up the entire RonDB cluster by terminating all the VMs created. After this command is executed all VMs and their data is lost.

Configuring a deployment#

Here is an example of the config.py which is used to configure the cluster setup.

# This config file is a python script. Only the config variable in the end
# matters, the rest are for convenience.

# For RonDB, we need the RonDB version and the glibc version.
latest_rondb = {
    "glibc_version": "2.28",
    "rondb_version": "24.10.7",
}

cluster_size = {
# The number of availability zones to use.
# Numbers larger than 1 means multi-AZ environment and 1 means single-AZ.
    "num_azs": 2,

# For ndbmtd, mysqld, rdrs and bench nodes, we can specify the number of VMs
# to use.
    "ndbmtd_count": 2,
    "mysqld_count": 2,
    "rdrs_count": 2,
    "bench_count": 2,
# We also need to specify the number of data node replicas.
# Note that replicas * node groups = data nodes.
# Therefore ndbmtd_count must be divisible by rondb_replicas.
    "rondb_replicas": 2,
# Disk sizes in GiB
    "ndbmtd_disk_size": 200,
    "mysqld_disk_size": 60,
    "rdrs_disk_size": 60,
    "prometheus_disk_size": 120,
    "bench_disk_size": 60,
}

# CPU platform and node types. All nodes will use the same CPU platform, either
# arm64_v8 or x86_64.
arm_config = {
    "cpu_platform": "arm64_v8",
    "ndb_mgmd_instance_type": "c8g.large",
    "ndbmtd_instance_type": "c8g.2xlarge",
    "mysqld_instance_type": "c8g.large",
    "rdrs_instance_type": "c8g.2xlarge",
    "prometheus_instance_type": "c8g.medium",
    "grafana_instance_type": "c8g.medium",
    "bench_instance_type": "c8g.2xlarge",
}
x86_config = {
    "cpu_platform": "x86_64",
    "ndb_mgmd_instance_type": "t3.large",
    "ndbmtd_instance_type": "t3.2xlarge",
    "mysqld_instance_type": "t3.large",
    "rdrs_instance_type": "t3.2xlarge",
    "prometheus_instance_type": "t3.medium",
    "grafana_instance_type": "t3.medium",
    "bench_instance_type": "t3.2xlarge",
}

config= {

    # AWS region
    "region": "eu-north-1",

    # RonDB version
    **latest_rondb,

    # Cluster size.
    **cluster_size,

    # Node type configs.
    **arm_config,
    #**x86_config,
}

The config at the bottom of the file defines the parameter sections. In this file we use the arm_config, the region we use is the eu-north-1 which is the swedish region in AWS. The latest_rondb section is used to calculate the name of the RonDB binary tarball to install. The cluster_size section is used to define the number of VMs of the various types, the number of Availability Zones to use, the disk size to use in the VMs.

Instance configuration#

In this case we need to define the instances used by the various VM types. However first we need to set the cpu_platform to either arm64_v8 or . This is used to make sure we get the correct RonDB binary tarball as well.

Obviously we need to ensure that the VM types we choose exist in AWS and that they actually use the CPU platform we configured. The instance types are defined by AWS. So e.g. c8g.2xlarge means a VM with 8 CPUs of type Graviton 4 with 16 GB of memory and up to 15Gb/sec of network bandwidth.

Cluster Size#

We make use of the following VM types:

  • ndb_mgmd, RonDB Management server

  • ndbmtd, RonDB Data node

  • mysqld, RonDB MySQL Server

  • rdrs, RonDB REST API Server (including Rondis)

  • prometheus, Prometheus time series database used by monitoring

  • grafana, Grafana, graphical UI to RonDB dashboards

  • bench, The benchmark clients

We can only have one Prometheus and one Grafana instance, thus they are not configurable and similarly we only support a single RonDB Management server. The other node types can be configured. We need at least one MySQL Server to be able to create tables, the number of RonDB REST API servers can be 0. Not having any benchmark clients would mean a cluster that cannot run any benchmarks, so probably should not use this tool.

The rondb_replicas is the number of replicas of the data. In a production cluster we would differ between this number and the number of active data replicas. In this tool we are currently not setting things up for flexibility, it is mainly set up for testing and experimentation.

By setting num_azs to e.g. 2 we ensure that we will spread the VMs on more than Availability Zone.

Network Load Balancer#

The tool will always set up network load balancer for the MySQL Servers, the RonDB REST API server and the Rondis server. It will use the AWS Network Load balancer. In most benchmark executions this will react very quickly and scale up. However when running Gbytes per second through it one sometimes need to run a second benchmark run for the network load balancer to scale up.

Experimenting with RonDB configurations#

If one wants to try with different configurations the /textttcluster_ctl tool will set defaults for MySQL configuration, for RonDB REST API configuration, for Rondis configuration and for RonDB cluster configuration.

If some of those things need to be changed one should do it before running the install command. Remember that the can be applied to a subset of the node types if this change happens on a cluster already created. Thus the install command can be handled even after the cluster is created to change the configurations of MySQL Servers, RonDB REST API servers and Rondis servers.

The RonDB cluster configuration can be changed by logging in to the ndb_mgmd and go into the config_files directory and edit the config.ini file. After this one need to run the start command which will first stop the cluster and next start it again from scratch.