NDB management client ndb_mgm#
The NDB management client has a few useful commands. The management client is used to start and stop nodes, show cluster status, shutdown the cluster.
The commands to handle backups using the NDB management client was covered in the chapter on Backup in MySQL Cluster.
The management client is used to add new node groups to the cluster and can also drop node groups that are still empty. This is covered in a chapter on online configuration changes.
It is possible to set the cluster in single user mode, this means that only one API node is allowed to be active.
The management client can be used to change the filters for what gets printed in the cluster log. The DUMP command provides a possibility to print lots of debug messages to the cluster log, most of those are undocumented since they require fairly deep understanding of the cluster code to be useful.
The HELP command lists all the commands supported and HELP RESTART will provide specific help on the RESTART command.
For someone that wants to understand more about NDB internals there is a set of commands listed in HELP DEBUG that shows various ways of logging signals in the data nodes. In a special compile mode it is possible to inject errors in the data nodes for efficient testing of cluster.
There is a command to get rid of node ids that haven’t been freed through the command PURGE STALE SESSIONS.
All the management client commands are also available through the NDB Management API, this API is a C API.
The command client remembers the history of commands and these can be retrieved by using up arrow and down arrow.
The examples used here comes from using an early version of MySQL Cluster 7.6.4, the output from MySQL Cluster 7.5 is more or less the same and no changes of concepts and ideas have happened in the 7.6 version the NDB management client.
SHOW command#
The simplest command is SHOW that displays the configured nodes and their current status.
When a node isn’t started it lists the hosts that are allowed to connect.
When a node has started it lists the MySQL version it started upon. In addition it also lists node id and hostname connected from for connected nodes.
For NDB data nodes it also lists the node group each data node belongs to.
Below is an example output from a small cluster started from MTR where there is one NDB management server, 2 data nodes and 14 API nodes. In this case there are 2 MySQL servers started with 3 cluster connection each, thus 6 API nodes are connected from 2 MySQL servers. The remaining 8 API nodes are available for connections from localhost.
The output shown comes from an early version of MySQL Cluster 7.6.4, the output from MySQL Cluster 7.5 looks the same.
ndb_mgm> show
Cluster Configuration
---------------------
[ndbd(NDB)] 2 node(s)
id=1 @127.0.0.1 (mysql-5.7.20 ndb-7.6.4, Nodegroup: 0, *)
id=2 @127.0.0.1 (mysql-5.7.20 ndb-7.6.4, Nodegroup: 0)
[ndb_mgmd(MGM)] 1 node(s)
id=49 @127.0.0.1 (mysql-5.7.20 ndb-7.6.4)
[mysqld(API)] 14 node(s)
id=51 @127.0.0.1 (mysql-5.7.20 ndb-7.6.4)
id=52 @127.0.0.1 (mysql-5.7.20 ndb-7.6.4)
id=53 @127.0.0.1 (mysql-5.7.20 ndb-7.6.4)
id=54 @127.0.0.1 (mysql-5.7.20 ndb-7.6.4)
id=55 @127.0.0.1 (mysql-5.7.20 ndb-7.6.4)
id=56 @127.0.0.1 (mysql-5.7.20 ndb-7.6.4)
id=57 (not connected, accepting connect from localhost)
id=58 (not connected, accepting connect from localhost)
id=59 (not connected, accepting connect from localhost)
id=231 (not connected, accepting connect from localhost)
id=232 (not connected, accepting connect from localhost)
id=233 (not connected, accepting connect from localhost)
START/STOP commands#
NDB management client cannot stop and start processes. For this an agent is required such as used by the MySQL Cluster Manager. It can however stop processes. It can also restart a data node and a management server. Data nodes have an option to run with a special angel process. In this case the data node is started from the angel process. When the data node stops the angel process can restart the data node again.
Thus the management client can be used to start and stop data nodes and management servers after the programs are started. This is heavily used in the NDB test programs where we can control start and stop of programs completely from the test program once all the cluster programs have started.
The management client have no capability to start and stop application programs and especially MySQL server programs.
All START/STOP/RESTART commands start with a node id, if ALL is used instead of a node id the command applies to all nodes (data nodes and management servers, only data nodes for START command).
For example to RESTART node 2, use the following command:
START command#
The START command can be applied on a data node that have been started with the option -n. This option means that the data node program will start, but the data node will not start until instructed to do so.
It is possible to reach this state through the RESTART command as well. This makes it possible to stop a node and at some later time restart it again.
STOP command#
The STOP command is used when it is desired to stop the program completely in a node.
This command will perform a graceful shutdown. This means that an attempt will be made to ensure that no interference with user transactions happen. Transactions have a few seconds to complete before the transactions are aborted to continue the node shutdown.
The command STOP -a means a more abrupt stop where we will not even ensure that committed transactions are safe on durable media. The use of this is mostly for testing purposes and not for production purposes.
Normally a STOP command will be aborted if the stop of the node means that the entire cluster is shut down. In order to shut it down in even in this case use the command STOP -f.
RESTART command#
The restart command is very useful in situations where it is necessary to change the configuration or for some other reason one needs to restart a data node or a management server.
The command first does a STOP, this stop can be affected by the parameters -a and -f similar to the STOP command. The START is normally immediate after the STOP. But using the flag -n it is possible to return without restarting until a START command is executed.
There is an option to restart using the -i flag. This flag means that an initial node restart will be performed. This flag should be used with great care since it will remove the entire file system of the data node. The node will get all its data from other live nodes in the cluster.
SHUTDOWN command#
The SHUTDOWN command shuts down all data nodes and management servers using a graceful shutdown.
REPORT command#
The REPORT command can report backup status, memory usage and print the current event log.
Here is an example of the output from the three different reports that can be printed. The event log is cut since it can be quite long.
The memory report reports IndexMemory and DataMemory usage.
ndb_mgm> 2 REPORT backupstatus
Node 2: Backup not started
ndb_mgm> 2 report MemoryUsage
Node 2: Data usage is 1%(10 32K pages of total 950)
Node 2: Index usage is 1%(10 32K pages of total 950)
ndb_mgm> 2 REPORT EventLog
2017-09-20 14:11:48 Node 2: CM_REGREF from Node 1 to our Node 2. ..
2017-09-20 14:11:51 Node 2: CM_REGREF from Node 2 to our Node 2. ..
......
2017-09-20 15:06:22 Node 2: Node Sent Heartbeat to node = 1
ndb_mgm>
STATUS command#
The easiest way to explain the STATUS command is by showing an example. This example also gives examples of some of the other commands.
The example starts with the same cluster as the SHOW command displayed. Node 2 is one of the data nodes and it is up and running when the example starts. In this case the status is started and the version of the data node software is also provided.
Next we perform the RESTART command, as can be seen this command doesn’t return until the command is completed. We used RESTART -n to see how it looks when the node is not started. The status in this case is not started.
The next step is to start node 2 again. The command returns when the start have been initiated and we can check the status of the restart. In this case the status is starting with some additional information about the start phase it has reached.
When the start completes we will get a printout of this even though the management client has already returned.
ndb_mgm> 2 status
Node 2: started (mysql-5.7.19 ndb-7.6.4)
ndb_mgm> 2 restart -n
Node 2: Node shutdown initiated
Node 2: Node shutdown completed, restarting, no start.
Node 2 is being restarted
ndb_mgm> 2 status
Node 2: not started (mysql-5.7.19 ndb-7.6.4)
ndb_mgm> 2 start
Database node 2 is being started.
ndb_mgm> Node 2: Start initiated (version 7.6.4)
ndb_mgm> 2 status
Node 2: starting (Last completed phase 100) (mysql-5.7.19 ndb-7.6.4)
ndb_mgm> Node 2: Started (version 7.6.4)
ndb_mgm>
PROMPT command#
In some cases it might be useful to have several windows with management clients and it might be necessary to change the command prompt. The PROMPT command changes the prompt.
ENTER SINGLE USER MODE command#
ENTER SINGLE USER MODE node_id is a command that can be useful if it is required to perform an offline ALTER TABLE change.
When single user mode is entered, only the API node that was given in the command is allowed access to the cluster. Normally changes in the cluster are performed using online operations. This command provides a safe way of performing bigger changes using ALTER TABLE that cannot be performed while updating the cluster. It leaves one MySQL Server with access to the cluster, this can be used for read while the ALTER TABLE is performed.
After completing the offline operation one goes back to normal mode by using the command EXIT SINGLE USER MODE.
CLUSTERLOG command#
The CLUSTERLOG command controls what log messages that will end up in the cluster log.
The commands CLUSTERLOG ON, CLUSTERLOG OFF and CLUSTERLOG TOGGLE are used to change the on and off status of the various severity reports of the cluster log.
The severities are DEBUG, INFO, WARNING, ERROR, CRITICAL, ALERT.
ON sets it to ON, OFF sets it to OFF and TOGGLE changes the state from ON to OFF or from OFF to ON dependent on the state at the time of the command.
This command changes what is reported by the management server. By default all severities except DEBUG is reported.
In addition we have a number of log categories. These are STARTUP, SHUTDOWN, STATISTICS, CHECKPOINT, NODERESTART, CONNECTION, ERROR, INFO, CONGESTION, DEBUG, and BACKUP.
Each of those categories can have their own log level that can be set with a CLUSTERLOG command.
We provide an example where we set the log level of node restart and backup to 8.
ndb_mgm> 2 clusterlog noderestart=8 backup=8
Executing CLUSTERLOG NODERESTART=8 on node 2 OK!
Executing CLUSTERLOG BACKUP=8 on node 2 OK!
ndb_mgm>
DUMP command#
The DUMP command is mentioned when using the HELP DEBUG command. It can activate printouts to the cluster log, to the NDB data node log and it can even perform some intrusive changes of variables in the code to perform workarounds of difficult problems.
This command is not normally used by developers or DBAs, it is mostly intended for MySQL support that can use this to gather information about a special issue. The DUMP commands forms a signal sent to various blocks in the data nodes. The code in those blocks to handle those signals is the documentation of what can be done using the DUMP command.
EXIT/QUIT command#
The EXIT command exits from the NDB management client as well as the QUIT command.
CONNECT command#
The CONNECT command can be used to reconnect to the management server or to connect to a different management server. The parameter to the CONNECT command is a connect string, thus host[:port].
This command can be useful to see if all management servers agree on the cluster status.
PURGE STALE SESSIONS#
In the history of MySQL Cluster there was at times issues that a node id was kept allocated for longer than it was supposed to. As a workaround for those situations the PURGE STALE SESSIONS command was developed. This purges any sessions from node ids that are no longer used.
It should hopefully not be needed, but it still exists.