Release Notes RonDB 21.04.6#
RonDB 21.04.6 is the sixth release of RonDB 21.04.
RonDB is based on MySQL NDB Cluster 8.0.23. It is a bug fix release based on RonDB 21.04.5.
RonDB 21.04.6 is released as an open source SW with binary tarballs for usage in Linux usage. It is developed on Linux and Mac OS X and using WSL 2 on Windows (Linux on Windows).
RonDB 21.04.6 can be used with both x86_64 and ARM64 architectures although ARM64 is still in beta state.
The RonDB 21.04.6 is tested and verified on both x86_64 and ARM platforms using both Linux and Mac OS X. It is however only released with a binary tarball for x86_64 on Linux.
There are four ways to use RonDB 21.04.6:
You can use the managed version available on hopsworks.ai. This sets up a RonDB cluster provided a few details on the HW resources to use. The RonDB cluster is integrated with Hopsworks and can be used for both RonDB applications as well as for Hopsworks applications. Currently AWS is supported, Azure support is soon available.
You can use the cloud scripts that will enable you to set up in an easy manner a cluster on Azure or GCP. This requires no previous knowledge of RonDB, the script only needs a description of the HW resources to use and the rest of the set up is automated.
You can use the open source version and use the binary tarball and set it up yourself.
You can use the open source version and build and set it up yourself. This is the commands you can use:
# Download x86_64 on Linux wget https://repo.hops.works/master/rondb-21.04.6-linux-glibc2.17-x86_64.tar.gz # Download ARM64 on Linux wget https://repo.hops.works/master/rondb-21.04.6-linux-glibc2.31-arm64_v8.tar.gz # Download x86_64 on Mac OS X (at least Mac OS X 11.6) wget https://repo.hops.works/master/rondb-21.04.6-macosx11.6-xcode-13.1-x86_64.tar.gz # Download ARM64 on Mac OS X (at least Mac OS X 12.3) wget https://repo.hops.works/master/rondb-21.04.6-macosx12.3-xcode-13.1-arm64_v8.tar.gz
RonDB 21.04 is a Long Term Support version that will be maintained until at least 2024.
Maintaining 21.04 means mainly fixing critical bugs and minor change requests. It doesn't involve merging with any future release of MySQL NDB Cluster, this will be handled in newer RonDB releases.
Backports of critical bug fixes from MySQL NDB Cluster will happen.
Summary of changes in RonDB 21.04.6#
RonDB has 6 bug fixes since RonDB 21.04.5. In total RonDB 21.04 contains 15 new features on top of MySQL Cluster 8.0.23 and a total of 102 bug fixes.
RonDB has a number of unit tests that are executed as part of the build process.
RonDB has a functional test suite using the MTR (MySQL Test Run) that executes more than 500 RonDB specific test programs. In adition there are thousands of test cases for the MySQL functionality. MTR is executed on both Mac OS X and Linux.
We also have a special mode of MTR testing where we can run with different versions of RonDB in the same cluster to verify our support of online software upgrade.
RonDB is very focused on high availability. This is tested using a test infrastructure we call Autotest. It contains also many hundreds of test variants that takes around 36 hours to execute the full set. One test run with Autotest uses a specific configuration of RonDB. We execute multiple such configurations varying the number of data nodes, the replication factor and the thread and memory setup.
An important part of this testing framework is that it uses error injection. This means that we can test exactly what will happen if we crash in very specific situations, if we run out of memory at specific points in the code and various ways of changing the timing by inserting small sleeps in critical paths of the code.
During one full test run of Autotest RonDB nodes are restarted thousands of times in all sorts of critical situations.
Autotest currently runs on Linux with a large variety of CPUs, Linux distributions and even on Windows using WSL 2 with Ubuntu.
We test RonDB using the Sysbench test suite, DBT2 (an open source variant of TPC-C), flexAsynch (an internal key-value benchmark), DBT3 (an open source variant of TPC-H) and finally YCSB (Yahoo Cloud Serving Benchmark).
The focus is on testing RonDBs LATS capabilities (low Latency, high Availability, high Throughput and scalable Storage).
Finally we also execute tests in Hopsworks to ensure that it works with HopsFS, the distributed file system built on top of RonDB, and HSFS, the Feature Store designed on top of RonDB, and together with all other use cases of RonDB in the Hopsworks framework.
RONDB-126: Fix neighbour handling#
The fix is built on top of the previous fix of neighbour nodes where we introduced the function insert_activate_trp to insert transporters that have been put in list as neighbours and now need to be inserted in list of transporters for non-neighbours.
The code disables all neighbour transporters and moves them to non-neighbour status in startChangeNeighbourNodes. The change is that startChangeNeighbourNodes calls insert_activate_trp for all transporters moved to non-neighbour status. This avoids a small probability that a signal might be lost in the change from neighbour transporter to non-neighbour transporter.
We also call flush_send_buffers to ensure that any signals sent is visible to sender.
We also ensured that all nodes in the same node group always are neighbour nodes.
RONDB-128: List management bug in setNeighbourNode#
When enabling a node as neighbour node in setNeighbourNode the transporters belonging to the node are removed from the list of ready non-neighbour transporters. This code missed updating the last pointer and also lost any transporters in the list before the transporter being moved to neighbour. This caused that node to stop communicating.
RONDB-124: Concurrent release of same page possible in Suma#
When we enter into an out of buffer release process we can still simultaneously get a FIRE_TRIG_ORD that discovers also that some pages need to be released. This should be avoided when out of buffer release is ongoing since that will release all pages anyways.
Fixed by introducing a state variable whether out of buffer release is ongoing or not.
Bug33856371: Decreased overhead of SEND_PACKED#
Backport from MySQL 8.0.29.
Bug33872577: Avoid running out of job buffers in main thread#
Backport from MySQL 8.0.29.
Bug33797931: Improvement of BLOB operations in NDB API#
Backport from MySQL 8.0.29.