Release Notes RonDB 24.10.0#
RonDB 24.10.0 is the first official release of the RonDB 24.10 series.
RonDB 24.10.0 is based on MySQL NDB Cluster 8.4.3 and RonDB 22.10.10.
RonDB 21.04 is a Long-Term Support version of RonDB that will be supported for a few months more.
RonDB 22.10 is a Long-Term support version and will be maintained at least until 2025.
RonDB 24.10 will be a Long-Term support version and will be maintained at least until 2027.
RonDB 24.10 is released as open source SW with binary tarballs for usage in Linux. It is developed on Linux and Mac OS X and using WSL 2 on Windows (Linux on Windows).
RonDB 24.10.0 and onwards is supported on Linux/x86_64 and Linux/ARM64.
The other platforms are currently for development and testing. Mac OS X is a development platform and will continue to be so.
Description of RonDB#
RonDB is designed to be used in a managed cloud environment where the user only needs to specify the type of the virtual machine used by the various node types. RonDB has the features required to build a fully automated managed RonDB solution.
It is designed for appplications requiring the combination of low latency, high availability, high throughput and scalable storage (LATS).
You can use RonDB in a Serverless version on app.hopsworks.ai. In this case Hopsworks manages the RonDB cluster and you can use it for your machine learning applications. You can use this version for free with certain quotas on the number of Feature Groups (tables) you are allowed to add and quotas on the memory usage. You can get started in a minute with this, no need to setup any database cluster and worry about its configuration, it is all taken care of.
You can use the managed version of RonDB available on hopsworks.ai. This sets up a RonDB cluster in your own AWS, Azure or GCP account using the Hopsworks managed software. This sets up a RonDB cluster provided a few details on the HW resources to use. These details can either be added through a web-based UI or using Terraform. The RonDB cluster is integrated with Hopsworks and can be used for both RonDB applications as well as for Hopsworks applications.
You can use the cloud scripts that will enable you to set up in an easy manner a cluster on AWS, Azure or GCP. This requires no previous knowledge of RonDB, the script only needs a description of the HW resources to use and the rest of the set up is automated.
You can use the open source version and use the binary tarball and set it up yourself.
You can use the open source version and build and set it up yourself.
This is the commands you can use to retrieve the binary tarball:
# Download x86_64 on Linux
wget https://repo.hops.works/master/rondb-24.10.0-linux-glibc2.28-x86_64.tar.gz
# Download ARM64 on Linux
wget https://repo.hops.works/master/rondb-24.10.0-linux-glibc2.28-arm64_v8.tar.gz
These versions are also available as Docker containers at
hub.docker.com under hopsworks/rondb. See
https://hub.docker.com/r/hopsworks/rondb/tags for an up to date of
which RonDB container images are available. These containers can be used
to run RonDB in containers, in Hopsworks they are also used since
version 4.0 as containers in the Hopsworks Kubernetes cluster.
The actions to build both /textttx86_64 and ARM64 tarballs have now
been fully automated.
Summary of changes in RonDB 24.10.0#
RonDB 24.10.0 is based on MySQL NDB Cluster 8.4.3 and RonDB 22.10.10.
RonDB 24.10.0 adds 11 new features on top of RonDB 22.10.10, and adds 63 new features on top of MySQL NDB Cluster 8.4.3. In addition it adds a new product, the REST API Server that can be used to access data in RonDB using a REST protocol, this includes an endpoint for RonSQL that can be used to handle optimised aggregation queries. It also includes a prototype of Rondis, a server accepting parts of the Redis protocol.
Test environment#
RonDB uses four different ways of testing. MTR is a functional test framework built using SQL statements to test RonDB.
The Autotest framework is specifically designed to test RonDB using the NDB API. The Autotest is mainly focused on testing high availability features and performs thousands of restarts using error injection as part of a full test suite run.
Benchmark testing ensures that we maintain the throughput and latency that is unique to RonDB. The benchmark suites used are integrated into the RonDB binary tarball making it very straightforward to run benchmarks for RonDB.
Finally we also test RonDB in the Hopsworks environment where we perform both normal actions as well as many actions to manage the RonDB clusters.
RonDB has a number of MTR tests that are executed as part of the build process to improve the performance of RonDB.
MTR testing#
RonDB has a functional test suite using the MTR (MySQL Test Run) that executes more than 500 RonDB specific test programs. In adition there are thousands of test cases for the MySQL functionality. MTR is executed on both Mac OS X and Linux.
We also have a special mode of MTR testing where we can run with different versions of RonDB in the same cluster to verify our support of online software upgrade.
Autotest#
RonDB is very focused on high availability. This is tested using a test infrastructure we call Autotest. It contains also many hundreds of test variants that takes around 36 hours to execute the full set. One test run with Autotest uses a specific configuration of RonDB. We execute multiple such configurations varying the number of data nodes, the replication factor and the thread and memory setup.
An important part of this testing framework is that it uses error injection. This means that we can test exactly what will happen if we crash in very specific situations, if we run out of memory at specific points in the code and various ways of changing the timing by inserting small sleeps in critical paths of the code.
During one full test run of Autotest, RonDB nodes are restarted thousands of times in all sorts of critical situations.
Autotest currently runs on Linux with a large variety of CPUs, Linux distributions and even on Windows using WSL 2 with Ubuntu.
Benchmark testing#
We test RonDB using the Sysbench test suite, DBT2 (an open source variant of TPC-C), flexAsynch (an internal key-value benchmark), DBT3 (an open source variant of TPC-H) and finally YCSB (Yahoo Cloud Serving Benchmark). We test the REST API server using benchmark programs written in Go and Feature Store REST API server is benchmarked using a tool called Locust that can be used to test various application variants. This tool is heavily used to benchmark application scenarios required by Hopsworks customers.
Dydra, a community user of RonDB implements a graph database supporting SPARQL on top of RonDB using a Common Lisp NDB API. They also have a set of benchmarks to verify the performance of RonDB in their application.
The focus is on testing RonDBs LATS capabilities (low Latency, high Availability, high Throughput and scalable Storage).
Hopsworks testing#
Finally we also execute tests in Hopsworks to ensure that it works with HopsFS, the distributed file system built on top of RonDB, and HSFS, the Feature Store designed on top of RonDB, and together with all other use cases of RonDB in the Hopsworks framework.
New features#
RONDB-861: Added many new interpreter functions#
RonDB has an interpreter that can be used to execute simple programs using a native language. The program is defined in the NDB API and sent with the request. The virtual machine has 8 registers that can store 64-bit signed integers. There are instructions to add, subtract, multiply, divide, shift left, shift right, NO, AND, OR and XOR bitwise operations. One can load constants into the register of various sizes.
A memory has also been introduced into the interpreter, this memory is 64 kByte in size.
One can load larger memory into this interpreter memory. This memory can be used to read columns (VARBINARY columns for interpreter memory), write columns, also read and write partial is possible and also appending to a VARBINARY column.
There are instructions to read from memory into a register and write to a register from memory.
Writing into VARBINARY columns requires some instructions to handle conversion of sizes into little-endian format.
There are instructions to send results back to the NDB API, there are also instructions to read input from NDB API in parammeters. This makes it possible to reuse an interpreter program with different parameters.
There is a large range of branch instructions and these instructions need a label defined to jump to if the branch condition is true.
There are also some library functions such as, string length, converting string to 64-bit signed integer, converting from 64-bit signed integer to string, binary search using a sorted array of numbers. There are 7 different sizes of the numbers supported and 5 different binary search algorithms supported. Based on the binary search algorithm it is easy to make searches in a sorted array of intervals. There are instructions to do this for 7 different number sizes and two different range types.
To support this there is a qsort instruction and an instruction to compress 8-byte number arrays to 5-byte and 6-byte number arrays and similarly from 4-byte number arrays compressed to 3 byte arrays.
Finally there is an instruction to search for a specific string. This instruction is not taking character sets into account, it is simply making a simple memory compare.
RONDB-832: Rondis server#
As part of the REST API server (RDRS2) we have also added a Redis server prototype. Rondis supports the following commands:
-
GET
-
SET
-
DEL
-
INCR
-
DECR
-
INCRBY
-
DECRBY
-
GETRANGE
-
SETRANGE
-
STRLEN
-
HGET
-
HSET
-
HDEL
-
HINCR
-
HDECR
-
HINCRBY
-
HDECRBY
-
PING
Rondis also supports a number of commands that sets and gets multiple rows. These are currently treated as a number of independent gets and sets.
-
MGET
-
MSET
-
HMGET
-
HMSET
There are a few configuration parameters, it will be possible to define the behaviour of these commands through these configuration parameters. These configuration parameters are set per database, so the same Rondis can serve multiple applications with different requirements.
As an example we support a dirty flag for increment and decrement operations. Since RonDB is a distributed DBMS with two-phase commits it is possible to update a single row a few thousand times per second. If it is necessary to increment a counter faster than that one can use a dirty flag, this will ensure that Rondis can support hundreds of thousands of increments and decrements per second.
RDRS2 can run both a Rondis server and a REST API server at the same time. They will serve different ports, but they will use the same connections to the RonDB data nodes.u It is possible to disable and enable those to avoid this sharing.
Using the new Rate limit and Quota feature one can limit both the size of a Rondis database as well as the amount of traffic it is allowed to handle.
Rondis reuses the new TTL feature in RonDB to implement TTL support for Rondis. The TTL purger also runs in the RDRS2 binary.
See the documentation for some examples of performance of Rondis.
New REST API Server#
A major development is the new REST API Server that is now implemented in C++. It conforms to the same protocol as the REST API Server in 22.10 which was implemented in Go. Performance has been significantly improved, much of this comes from the use of the simdjson library that is about 30x faster on JSON parsing compared to the Go implementation. The performance improvement varies between 25% up to 900%. See the documentation for some benchmark examples.
The new REST API Server also implements an endpoint for RonSQL that is providing much faster aggregation queries. As mentioned it also implements the TTL purge service and Rondis. More about these in separate sections.
We removed support for GRPC in new REST API server.
The REST API Server also exports a port to read Prometheus data about latency for requests to REST API Server, RonSQL and Rondis.
RONDB-753: TTL feature#
In RonDB 24.10 one can now define a time-to-live for rows in a table. The TTL is defined on a DATETIME or TIMESTAMP column. The RonDB data nodes will hide rows that are expired. These rows can still be seen using special flags in the request. The expired rows are purged by a TTL purge service that runs in all RDRS2 servers. Thus to use the TTL service it is required to have a set of REST API servers.
This feature is natural in combination with the improved support for aggregation. RonDB 24.10 supports efficient aggregate queries and TTL makes it possible to let rows stay in the database for a specific time, thus we store a window of features that is dynamically moving and we can calculate any aggregates on this time window and thus get online aggregate results which will improve result of the inference queries.
Pushdown aggregation#
Aggregate queries were previously handled by the MySQL Server. This meant that the row data had to be sent to the MySQL Server for all rows. By supporting pushdown aggregates we perform the aggregation in the RonDB data nodes instead. Currently this is supported for aggregates on a single table.
With this feature the aggregation is handled by a special aggregation interpreter in the RonDB data nodes. The feature can be used from the NDB API, but we expect that most users will access it from RonSQL. There is no support for using this feature from MySQL Server.
The Q1 query in TPC-H see a speed up of more than 20x by using this feature through RonSQL.
RonSQL#
A new SQL server is implemented in the REST API server. This means that you can send SQL queries using an endpoint in the REST API server. Currently RonSQL only supports aggregation queries on a single table.
RONDB-503: Rate limits and Quotas#
This feature adds the possibility to limit memory and disk usage for a database in RonDB and similarly one can also limit the usage of (rate limit) a database in RonDB. This includes limiting both the number of concurrent transactions and the size of transactions.
RONDB-820: Improved recovery by parallelising a number of algorithms#
This feature is a speed up by a factor of 2 of restarts with many tables by doing more things in parallel in the restart phases.
RONDB-780: Increased possible batch size on scan queries from 992 to 4096#
RONDB-785: Added new command to set LocationDomainId in ndb_mgm#
RONDB-789: Automatic memory management when running in a container#
Bug Fixes#
RONDB-855: Make sure that ACTIVATE and DEACTIVATE executes in an idempotent manner#
RONDB-842: Ensure correct order of UNDO log files in recovery#
It was possible to get the order of UNDO log files in correct if a crash happened fairly soon after the UNDO log file was created. This fixes that problem.