Skip to content

Building RonDB from source#

All the scripts to build a production release of RonDB is in the github tree. We build RonDB in a docker container to ensure that we have a standardised build system for release builds.

However if you are anything like me, you want to develop and build in a normal terminal. This requires you to install all the tools that we install using our docker container.

The benefit of this approach is that the builds are so much faster. A complete rebuild of RonDB takes about 5-10 minutes on a modern laptop or desktop computer. If one only changed small parts of the code, it will only take a few seconds or at most 1-2 minutes to rebuild. The complete rebuild of a release build of RonDB takes around 3 hours, so a very large difference.

The best documentation of how to build RonDB on Linux is to read the Dockerfile for the distro you are using. Currently we build in 3 different distros, Oracle Linux 8, Oracle Linux 9 and Ubuntu 22.04. The Dockerfile used for our production builds of x86_64 and ARM64 is using Oracle Linux 8, thus this Dockerfile is currently most likely to mimic the exact current build requirements.

If you are a developer and work on Mac OS X I will describe the essential parts to prepare for building RonDB on Mac OS X.

Once you have prepared your machine for building RonDB the below commands show how you can compile your own development version of RonDB. I assume you use a directory called rondb_trees where you have your RonDB github trees. This particular tree we call rondb_src. Within the source directory create a directory called debug_build, RonDB uses out-of-source builds. Place yourself in this directory and from there run the build script build_script_debug.sh. This will build a debug version of RonDB. There is a version we use for Autotest (testing node failures) and tests that use error injection. This script is called build_script_err.sh. Another script builds an optimised version of RonDB (build_script_prod.sh) and finally there is a script that can be used to install RonDB in a specific directory called build_script_prefix.sh.

cd rondb_trees
git clone https://github.com/logicalclocks/rondb rondb_src
cd rondb_src
mkdir debug_build
cd debug_build
../build_scripts/build_script_debug.sh

After building RonDB you might want to test your changes. I usually start with a simple test called ndb_basic.

cd mysql-test
./mtr --suite=ndb ndb_basic

When your code has stabilised such that you want to try it with the full RonDB test suite, you can issue the following command. It will run all 11 test suites with many hundreds of different test cases. This is the functional test suite. There is also a test suite called Autotest, this tests a lot more of the high availability parts of RonDB and the full test suite takes 36 hours to run.

./mtr --suite=ndb,ndbcluster,ndb_binlog,ndb_ddl,ndb_opt,ndb_rpl,rpl_ndb,
              gcol_ndb,json_ndb,rdrs2,ronsql --parallel=4 --force

Build preparations#

At first to build RonDB you need a C++ compiler. On Linux we currently use the latest GCC 12 compiler. To install this on Linux use the following command (All those commands need to be executed with sudo user, or as a root user):

# Oracle Linux
yum install gcc-toolset-12 (gcc-12 g++-12 on Ubuntu)

Before doing this however it might be necessary to prepare yum such that it finds all the packages you need. This code is from the Dockerfile for Oracle Linux 8. Something similar to this might have to be done before starting to use yum install.

If you are building on Mac OS X, usually one need to use brew install in place of yum install. Obviously package names will differ, but the idea is the same. To build on Mac OS X the first step is to install XCode and its terminal tools. We are currently using version 15 and 16 of XCode and both works. Note that Mac OS X is only a development platform, it is not intended for production installations.

It is also possible to use RonDB on Windows, but only in WSL 2, that is running Linux on Windows. Actually we use this alternative to run many of our test suites that run in background since Microsoft Remote Desktop makes it easy to work with Windows laptops in the background.

There is a set of tools that you need to install from source as well. You need to install and build a recent bison tool, at least version /texttt3.8. You need to install CMake, at least version 3.23.2. In the Dockerfiles we build CMake from source, this is not always a requirement if a recent version is available.

An important part of the preparations is to build OpenSSL. It is most often required to build this from source to get a recent version of OpenSSL. RonDB 24.10 requires the version to be at least OpenSSL 3.0, the later version, the better.

Next you need to install go, this needs to be at least version 1.20.3.

Next you need to build from source jsoncpp, at least version 1.9.5. You also need to build Apache Avro from source, at least version 1.12.0.

Installing Boost is a requirement in earlier versions of RonDB, but from RonDB 24.10 it is no longer required. It is however required to build Apache Avro.

Now a long list of tools need to be installed as well using yum:

  1. make

  2. wget

  3. git

  4. which

  5. perl-core

  6. openldap-devel (libldap-dev on Ubuntu)

  7. flex

  8. krb5-server

  9. krb5-workstation

  10. krb5-devel

  11. numactl

  12. numactl-devel

  13. numactl-libs

  14. uuid-devel (libudev-dev on Ubuntu, libuuid-devel on RH 9)

  15. libudev-devel

  16. doxygen

  17. patchelf

  18. ncurses-devel (libncurses5-dev on Ubuntu)

  19. zlib-devel (zlib1g-dev on Ubuntu)

  20. scl-utils

  21. vim

  22. pigz

  23. hostname

  24. maven

  25. rpcgen

  26. bzip2

  27. bzip2-libs

  28. automake

  29. autoconf (on Ubuntu)

  30. pkg-config (on Ubuntu)

  31. build-essential (on Ubuntu)

  32. checkinstall (on Ubuntu)

  33. libsasl2-dev (on Ubuntu)

  34. libaio-dev (on Ubuntu)

  35. protobuf-compiler (on Ubuntu)

  36. openssh-client (on Ubuntu)

  37. graphviz (on Ubuntu)

To build ClusterJ one also need to install Java 8 for the build. One can run with newer Java versions, but Java 8 is used for the build.

  1. default-jdk (on Ubuntu)

  2. java-1.8.0-openjdk-devel (Oracle Linux 8)

In addition Java requires some setup of a Maven repository as this portion of the Dockerfile shows:

COPY <<-"EOF" /root/.m2/settings.xml
<settings xmlns="http://maven.apache.org/SETTINGS/1.0.0"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.0.0 http://maven.apache.org/xsd/settings-1.0.0.xsd">
    <servers>
        <server>
            <id>${JenkinsHops.RepoID}</id>
            <username>${JenkinsHops.User}</username>
            <filePermissions>664</filePermissions>
            <directoryPermissions>775</directoryPermissions>
            <password>${JenkinsHops.Password}</password>
        </server>
    </servers>
</settings>
EOF

The actual release build script is placed in the build_scripts/release_scripts directory and the build script is called build_all.sh and uses a couple of other helper scripts.

There are also two Docker build scripts, docker-build.sh and docker-create-builder.sh that can be used to build using the Dockerfiles and using Docker.

Installing from a clean install of Rocky Linux 9.4#

The Dockerfiles are a good description of how to build RonDB. However every Linux distribution has some oddities to take care of. In this section we will describe a complete installation process after a clean install of Rocky Linux 9.4. There are essentially 3 OS platforms that we support building RonDB on, Red Hat-based distros (Red Hat, Oracle Linux, Alma Linux and Rocky Linux), Ubuntu based distros (too many to list) and Mac OS X where one as mentioned need to use at least XCode 15.

Installing build tools#

At first we need to install build tools and various tools needed to download and work with the source code, the following command accomplishes that.

sudo dnf install git cmake gcc-toolset-12 flex make tar which maven wget automake scl-utils

Building the latest RonDB version we use GCC version 12 for, it is possible to use newer versions of GCC as well, but we are very conservative on using the newest compiler versions to avoid as much as possible compiler bugs. RonDB is built using the C++ standard C++-17.

git is the version management tool we use, this is required to download the RonDB source code and to work with the code. RonDB uses CMake to create makefiles that are then used to build RonDB. Rocky Linux installed version 3.26.5 which is new enough to avoid having to download and build it from source.

Maven is the build tool used for the Java code in RonDB. automake will also ensure that autoconf is installed, this is used by some internal libraries that RonDB has as part of the source tree. scl-utils is a useful tool if you want to use more than one GCC version.

Installing development libraries#

Next we will install a large set of development libraries that the RonDB source code depends on. Most of those comes from being a fork of MySQL.

sudo dnf install rpcgen perl-core openldap-devel
sudo dnf install krb5-server krb5-workstation krb5-devel
sudo dnf install numactl numactl-libs numactl-devel
sudo dnf install libuuid-devel libudev-devel ncurses-devel
sudo dnf install java-1.8.0-openjdk-devel boost-devel
sudo dnf install bzip2 bzip2-libs zlib-devel
sudo dnf install vim pigz hostname

The MTR test framework in RonDB is built in Perl, so sometimes one might require installation of special parts of Perl, but mostly perl-core installs the necessary components.

There are libraries for LDAP security, libraries for controlling CPU bindings of processes that are used in some test cases, libraries to handle networking protocols, UUID handling and compression libraries and libraries to handle nice printouts.

RonDB is built with on old version of Java, Java 8, this is only for building, it is quite ok to use newer versions to execute the Java code. Newer versions of Java have many optimisations that are significant.

Before installing patchelf we need to install the extra tools. This tool is used to patch the compiled libraries.

sudo dnf install epel-release
sudo dnf install patchelf
sudo dnf install --enablerepo doxygen

Finally we also install doxygen that is used for documentation of APIs.

Building required components#

Some components of RonDB are external, but needs to be on the latest version and thus installing packages is not ideal. For these libraries we build them from source after downloading. Before starting we need to set a number of environment variables, we will use /tmp/downloads to build those libraries.

export DOWNLOADS_CACHE_DIR=/tmp/downloads
export OPENSSL_ROOT=/usr/local/ssl
export OPENSSL_ROOT_DIR=$OPENSSL_ROOT
export OPENSSL_VERSION=3.0.15
export GO_TAR_NAME=go1.23.2.linux-amd64.tar.gz
export AVRO_VERSION=1.12.0
export JSONCPP_VERSION=1.9.5

We need to build OpenSSL, Apache Avro, JsonCpp, and bison. We need to download Golang.

This is generally performed in a number of steps:

1. Use wget to download the tarball
2. Untar the tarball
3. cd to the build tree
4. Create a build directory
5. Configure the build
6. Compile using make
7. Install using make

Process for OpenSSL:

cd /tmp/downloads
wget https://github.com/openssl/openssl/releases/download/openssl-$OPENSSL_VERSION/openssl-$OPENSSL_VERSION.tar.gz
tar xf openssl-$OPENSSL_VERSION.tar.gz
cd openssl-$OPENSSL_VERSION
./config --prefix=$OPENSSL_ROOT --openssldir=$OPENSSL_ROOT shared zlib
make -j8
sudo make install

Process for Golang:

cd /tmp/downloads
wget https://golang.org/dl/$GO_TAR_NAME
sudo tar -C /usr/local -zxf $DOWNLOADS_CACHE_DIR/$GO_TAR_NAME
export PATH=$PATH:/usr/local/go/bin

Process for JsonCpp:

cd /tmp/downloads
wget https://github.com/open-source-parsers/jsoncpp/archive/refs/tags/$JSONCPP_VERSION.tar.gz
tar xf $JSONCPP_VERSION.tar.gz
cd json-$JSONCPP_VERSION
mkdir build
cd build
cmake -DCMAKE_BUILD_TYPE=release -DBUILD_SHARED_LIBS=ON -DARCHIVE_INSTALL_DIR=. -G "Unix Makefiles" -DCMAKE_INSTALL_PREFIX=/usr -DCMAKE_INSTALL_INCLUDEDIR=include/jsoncpp ..
make -j8
sudo make install

Process to build Apache Avro:

cd /tmp/downloads
wget https://github.com/apache/avro/archive/refs/tags/release-$AVRO_VERSION.tar.gz
tar xf release-$AVRO_VERSION.tar.gz
cd avro-release-$AVRO_VERSION/lang/c++
mkdir build
cd build
cmake -G "Unix Makefiles" ..
make -j8
sudo make install

Process to build /textttbison:

curl -LO https://ftp.gnu.org/gnu/bison/bison-3.8.2.tar.gz
tar xzf bison-3.8.2.tar.gz
cd bison-3.8.2
./configure --prefix=/
make -j8
sudo make install

After this long series of preparations we are now ready to build RonDB itself. Rocky Linux 9.4 will install a Boost version 1.75.0 and use this in building Apache Avro, one can also download and install a newer version of /textttBoost using the following process:

export BOOST_VERSION=1.86.0
export BOOST_V_UNDERSCORE=1_86_0
cd /tmp/downloads
wget https://archives.boost.io/release/$BOOST_VERSION/source/boost_$BOOST_V_UNDERSCORE.tar.bz2
tar --bzip2 -xf boost_$BOOST_V_UNDERSCORE.tar.bz2
cd boost_$BOOST_V_UNDERSCORE
./bootstrap.sh
./b2 -j8
./b2 install

After this we can free the disk space from /tmp/downloads:

cd /tmp/downloads
rm -rf *

Building RonDB#

We are now ready to build RonDB through the following commands:

cd
mkdir rondb_trees
cd rondb_trees
git clone https://github.com/logicalclocks/rondb rondb_src
cd rondb_src
git checkout 24.10-main
mkdir debug_build
cd debug_build
../build_scripts/build_script_debug.sh
cd mysql-test
./mtr --suite=ndb ndb_basic

The above process downloaded the latest RonDB version from RonDB and you have built it and even ran your first test program. If any problems occur in the process it is likely due to the various differences between Linux distros and Google and ChatGPT can hopefully provide you with the assistance required to resolve any issues you encounter. You can also contact our community from community.rondb.com.

You are now ready to develop additional features to RonDB, if you have patches already against MySQL these should apply fairly cleanly since RonDB haven’t changed so many parts of the MySQL Server code base. However the NDB storage engine is heavily changed in RonDB and there is also a new set of tools such as RonDB REST API server including RonSQL and a number of other goodies in development.