Skip to content

TLS Between Cluster Nodes#

RonDB can encrypt the connections between the nodes of a cluster with TLS. It uses TLS 1.3 with mutual authentication: both ends of a connection present a certificate signed by the certificate authority (CA) of the cluster. TLS between cluster nodes is available in RonDB 24.10 and later.

The REST API server has separate TLS settings for HTTPS between its clients and the REST API server, see REST API Server. This chapter covers the connections inside the cluster.

What TLS protects#

TLS can be used on these connections:

  • TCP connections between data nodes.

  • TCP connections between API nodes, such as MySQL Servers, and data nodes.

  • Connections to management servers, including the management protocol that ndb``_``mgm, MySQL Servers and the NDB tools use to fetch the configuration.

Shared memory (SHM) connections and RDMA connections are never encrypted.

Keys and certificates#

The tool ndb``_``sign``_``keys creates the CA, the private keys and the certificates.

The CA consists of the files NDB-Cluster-cert and NDB-Cluster-private-key. The CA private key is protected by a passphrase of at least 4 characters. The CA certificate is valid for 4 years.

Each host has one private key and one certificate for each type of node that runs on it: ndb-data-node-private-key and ndb-data-node-cert for data nodes, and the files beginning with ndb-mgm-server- and ndb-api- for management servers and API nodes. The certificate file also contains the CA certificate, so the CA files are not needed on the hosts of the nodes. The private keys are not encrypted and are created readable only by their owner.

Node certificates are valid for a few months: 150 days for management servers, 130 to 140 days for data nodes and 120 to 130 days for API nodes. The certificates of management servers and API nodes contain the HostName of the node from the cluster configuration. When such a node connects to a data node, the data node checks that the connection comes from an address that this hostname resolves to. Data node certificates are not bound to a hostname. A certificate identifies the type of node, not the node id: the receiving node checks that the type in the certificate matches the type that the cluster configuration gives the connecting node id.

The nodes find their keys and certificates in the directories of the TLS search path, set with the option --ndb-tls-search-path. The default is $HOME/ndb-tls. The search path is a list of directories separated by colons, and it can refer to environment variables such as $HOME. A node checks at startup that its certificate is signed and has not expired.

Setting up TLS#

The following steps set up TLS for a cluster. The examples use /var/ndb-ca for the CA and /var/ndb-tls for the keys and certificates of the nodes.

First create the CA on a host that is kept secure. The tool asks for the passphrase:

ndb_sign_keys --create-CA --CA-search-path=/var/ndb-ca

Next, on each host, create the keys and certificates for the nodes that the cluster configuration places on that host. The tool reads the configuration file, finds the nodes whose HostName is this host, and signs their certificates with the CA:

ndb_sign_keys -f config.ini --create-key \
    --CA-search-path=/var/ndb-ca --ndb-tls-search-path=/var/ndb-tls

If the CA is kept on another host, --remote-CA-host makes the tool sign the certificates on that host over ssh.

Set the search path for all programs on each host in the [mysql``_``cluster] section of my.cnf:

[mysql_cluster]
ndb-tls-search-path=/var/ndb-tls

Finally, require TLS in the cluster configuration, and start the cluster:

[ndbd default]
RequireTls=true

[ndb_mgmd default]
RequireTls=true

Configuration parameters#

RequireTls#

Can be set for data nodes and management servers. A data node with RequireTls requires TLS on all its TCP connections. A management server with RequireTls requires TLS from all its clients, except for the few commands that start TLS. A node with RequireTls that has no valid key and certificate does not start. The default is false.

API nodes have no such parameter. A connection between an API node and a data node or management server uses TLS when the data node or management server requires it, and an API node without a valid certificate then cannot connect.

Set RequireTls on the management servers as well as on the data nodes. RequireTls on the data nodes alone does not make their connections to a management server without a certificate use TLS.

RequireTls applies to the connections that the management server creates from the cluster configuration, not to connections defined in explicit [TCP] sections.

RequireCertificate#

Can be set for data nodes and management servers. A node with RequireCertificate does not start without a valid key and certificate, but it does not require TLS on its connections. The default is false.

Program options#

  • --ndb-tls-search-path sets the TLS search path for ndbmtd, ndb``_``mgmd, ndb``_``mgm, mysqld and the NDB tools such as ndb``_``restore. It is best set in the [mysql``_``cluster] section of my.cnf.

  • --ndb-mgm-tls sets how a program connects to a management server. With relaxed, the default, it uses TLS when it has a certificate and the management server accepts TLS, and otherwise cleartext. With strict it must use TLS. ndb``_``mgm also accepts deferred, which connects in cleartext until the START TLS command.

  • ndb``_``mgm --test-tls connects with TLS required and exits, which checks the setup of a host.

In a MySQL Server both options are read-only system variables, ndb``_``tls``_``search``_``path and ndb``_``mgm``_``tls.

NDB API applications call Ndb``_``cluster``_``connection::configure``_``tls() before they connect, with the search path and 0 for relaxed or 1 for strict. Without this call, an application uses the certificates in the default search path for its connections to the data nodes, but connects to the management server in cleartext. ClusterJ uses the properties com.mysql.clusterj.tls.path and com.mysql.clusterj.tls.strict.

The REST API server connects to the management server in cleartext, so it cannot connect to a management server with RequireTls. For its connections to the data nodes it uses the certificates in the default search path, $HOME/ndb-tls of the user that runs it. RonDB Helm does not set up TLS between cluster nodes.

Monitoring#

  • The management client command TLS INFO shows whether the client’s connections use TLS, the TLS connections of the management server with their certificates, and connection statistics, see NDB management client.

  • The ndbinfo table certificates lists the certificates that the data nodes know: their own and those of the nodes connected to them with TLS, with name, serial number and expiry date.

  • The column encrypted of the ndbinfo table transporters is 1 for connections that use TLS.

  • At startup a data node or management server logs NDB TLS 1.3 available using certificate file ’...’, and a MySQL Server logs Using NDB TLS certificate file ’...’. A data node with RequireTls also logs This node will require TLS for all connections.

Replacing certificates#

Node certificates expire after a few months and must be replaced in time. A node reads its certificate only when it starts, so a new certificate takes effect when the node restarts.

ndb``_``sign``_``keys --check --replace-by=-N exits with code 1 when a certificate in the search path expires within N days:

ndb_sign_keys --no-config --check --replace-by=-15 \
    --ndb-tls-search-path=/var/ndb-tls

To replace them, create new keys and certificates as pending files with --pending, and make them active with --promote; the old files are kept as retired files. Then restart the nodes on the host, one at a time.

ndb_sign_keys -f config.ini --create-key --pending \
    --CA-search-path=/var/ndb-ca --ndb-tls-search-path=/var/ndb-tls
ndb_sign_keys -f config.ini --promote --ndb-tls-search-path=/var/ndb-tls

The CA itself is replaced with --rotate-CA. The new CA is signed by the old one, and certificates signed by the new CA also carry the old CA, so nodes with old and new certificates can connect to each other during the change.

Limits#

  • SHM and RDMA connections are not encrypted.

  • Nodes that have certificates still use cleartext between them unless TLS is required. Only connections to a management server use TLS whenever the connecting node has a certificate.

  • Certificates are valid from the time they are signed. A host whose clock is behind the host that signed a certificate can reject it as not yet valid.