Deploying a TiDB Cluster Using TiUP

This is the installation method used for real production environments.
Tags:

TiUP is a cluster operation and maintenance tool introduced in TiDB 4.0. TiUP provides TiUP Cluster, a cluster management component written in Golang. With TiUP Cluster, you can easily perform routine database operations such as deploying, starting, stopping, deleting, scaling, and upgrading TiDB clusters, and you can manage TiDB cluster parameters.

TiUP supports deploying TiDB, TiFlash, TiDB Binlog, TiCDC, and monitoring systems. This document introduces how to deploy TiDB clusters with various topologies.

Step 1. Prerequisites and Prechecks

Check the following documents.

Step 2. Deploy TiUP on the Control System

You can deploy TiUP on the control system by using either online deployment or offline deployment.

Install the TiUP Tool

1. Install the TiUP tool

First, run the following command in an online environment to install TiUP.

curl --proto '=https' --tlsv1.2 -sSf https://tiup-mirrors.pingcap.com/install.sh | sh

% curl --proto '=https' --tlsv1.2 -sSf https://tiup-mirrors.pingcap.com/install.sh | sh
% Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                Dload  Upload   Total   Spent    Left  Speed
100 6988k  100 6988k    0     0  3433k      0  0:00:02  0:00:02 --:--:-- 3442k
WARN: adding root certificate via internet: https://tiup-mirrors.pingcap.com/root.json
You can revoke this by remove /Users/user/.tiup/bin/7b8e153f2e2d0928.root.json
Successfully set mirror to https://tiup-mirrors.pingcap.com
Detected shell: zsh
Shell profile:  /Users/user/.zshrc
/Users/user/.zshrc has been modified to add tiup to PATH
open a new terminal or source /Users/user/.zshrc to use it
Installed path: /Users/user/.tiup/bin/tiup
===============================================
Have a try:     tiup playground
===============================================

2. Declare the global environment variables again

source .bash_profile

% source .bash_profile

3. Check whether TiUP is installed

which tiup

% which tiup
/Users/user/.tiup/bin/tiup

Online TiUP Cluster Deployment

1. Install the TiUP cluster component

tiup cluster

% tiup cluster
tiup is checking updates for component cluster ...timeout!
The component `cluster` version  is not installed; downloading from repository.
download https://tiup-mirrors.pingcap.com/cluster-v1.11.0-darwin-arm64.tar.gz 8.31 MiB / 8.31 MiB 100.00% 6.73 MiB/s
Starting component `cluster`: /Users/user/.tiup/components/cluster/v1.11.0/tiup-cluster
Deploy a TiDB cluster for production

Usage:
tiup cluster [command]

Available Commands:
check       Perform preflight checks for the cluster.
deploy      Deploy a cluster for production
start       Start a TiDB cluster
stop        Stop a TiDB cluster
restart     Restart a TiDB cluster

... omitted ...

Use "tiup cluster help [command]" for more information about a command.

If this is the first time it is run, you can see that it downloads from the server.

2. If TiUP Cluster is already installed, update the components to the latest version with the TiUP command as follows

tiup update --self && tiup update cluster

% tiup update --self && tiup update cluster
download https://tiup-mirrors.pingcap.com/tiup-v1.11.0-darwin-arm64.tar.gz 6.82 MiB / 6.82 MiB 100.00% 2.76 MiB/s
Updated successfully!
component cluster version v1.11.0 is already installed
Updated successfully!

If “Updated successfully!” is displayed as above, TiUP Cluster has been updated successfully.

3. Check the current version of TiUP Cluster

tiup --binary cluster

% tiup --binary cluster
/Users/user/.tiup/components/cluster/v1.11.0/tiup-cluster

Offline TiUP Cluster Deployment

To deploy a TiDB cluster in an offline environment using TiUP, proceed as follows.

1. Prepare the TiUP offline component package

To prepare a TiUP offline component package, you can use tiup mirror clone to get the offline component package.

1-1. Fetch the mirror with TiUP

Fetch the required components from a system that can access the internet.

tiup mirror clone tidb-community-server-${version}-linux-amd64 ${version} --os=linux --arch=amd64

When actually run, it proceeds as follows.

% tiup mirror clone tidb-community-server-v1.11.0-linux-amd64 v1.11.0 --os=linux --arch=amd64
Start to clone mirror, targetDir is tidb-community-server-v1.11.0-linux-amd64, source mirror is https://tiup-mirrors.pingcap.com, selectedVersions are [v1.11.0]
If this does not meet expectations, please abort this process, read `tiup mirror clone --help` and run again
Arch [amd64]
OS [linux]
PCC linux/amd64 v1.11.0 not found, using 1.0.1 instead.
alertmanager linux/amd64 v1.11.0 not found, using v0.17.0 instead.
blackbox_exporter linux/amd64 v1.11.0 not found, using v0.21.1 instead.
br linux/amd64 v1.11.0 not found, using v6.3.0 instead.

... omitted ...

download https://tiup-mirrors.pingcap.com/tispark-v2.4.1-any-any.tar.gz 22.36 MiB / 22.36 MiB 100.00% 8.30 MiB/s
download https://tiup-mirrors.pingcap.com/tiup-v1.11.0-linux-amd64.tar.gz 6.92 MiB / 6.92 MiB 100.00% 5.94 MiB/s
download https://tiup-mirrors.pingcap.com/tiup-linux-amd64.tar.gz 6.92 MiB / 6.92 MiB 100.00% 5.84 MiB/s

The command above creates a directory named tidb-community-server-${version}-linux-amd64 in the current directory. This directory contains the component packages needed to start the cluster.

1-2. Compress the component package with the tar command and transfer it to the control computer in the isolated environment
tar czvf tidb-community-server-${version}-linux-amd64.tar.gz tidb-community-server-${version}-linux-amd64

% tar czvf tidb-community-server-v1.11.0-linux-amd64.tar.gz tidb-community-server-v1.11.0-linux-amd64

The tidb-community-server-${version}-linux-amd64.tar.gz file created by compression is an independent offline environment package.

1-3. Customize an offline mirror or adjust the contents of an existing offline mirror

To adjust an existing offline mirror, for example by adding a new component version, perform the following steps.

When fetching an offline mirror, you can specify information such as components and versions through parameters to get an incomplete offline mirror.
For example, you can run the following command to fetch an offline mirror that contains only TiUP v1.10.0 and TiUP Cluster v1.10.0.

tiup mirror clone tiup-custom-mirror-v1.10.0 --tiup v1.10.0 --cluster v1.10.0

If you need components only for a specific platform, you can specify them with the --os or --arch parameters.

Referring to “1-2” above, transfer this incomplete offline mirror to the control system in the isolated environment.

In the isolated environment, check the current offline mirror path of the control system. If the TiUP tool is a recent version, run the following command to get the current mirror address.

tiup mirror show

% tiup mirror show
https://tiup-mirrors.pingcap.com

If the output of the command above says that the show command does not exist, you may be using an older version of TiUP.
In that case, check the contents of $HOME/.tiup/tiup.toml to find the current mirror address.

% cat ~/.tiup/tiup.toml
mirror = "https://tiup-mirrors.pingcap.com"

Record this mirror address. In the next step, ${base_mirror} is used to refer to this address.

Merge the incomplete offline mirror into the existing offline mirror. First, copy the keys directory of the current offline mirror to the $HOME/.tiup directory.

cp -r ${base_mirror}/keys $HOME/.tiup/

Then use the TiUP command to merge the incomplete offline mirror into the mirror in use.

tiup mirror merge tiup-custom-mirror-v1.10.0

After the steps above are complete, run the tiup list command to check the result. In this document’s example, the output of tiup list tiup and tiup list cluster shows that the corresponding components of v1.10.0 are available.

Deploy Offline TiUP Components

After transferring the package to the control system of the target cluster, run the following command to install the TiUP components.

tar xzvf tidb-community-server-${version}-linux-amd64.tar.gz && \
sh tidb-community-server-${version}-linux-amd64/local_install.sh && \
source /home/tidb/.bash_profile

The local_install.sh script automatically runs the tiup mirror set tidb-community-server-${version}-linux-amd64 command and sets the current mirror address to tidb-community-server-${version}-linux-amd64.

To switch the mirror to another directory, run tiup mirror set <mirror-dir>. To switch the mirror to an online environment, run tiup mirror set https://tiup-mirrors.pingcap.com.

Step 3. Initialize the Cluster Topology File

Run the following command to create the cluster topology file.

tiup cluster template > topology.yaml

% tiup cluster template > topology.yaml
tiup is checking updates for component cluster ...
Starting component `cluster`: /Users/user/.tiup/components/cluster/v1.11.0/tiup-cluster template

In the following two common scenarios, you can run commands to create the recommended topology templates.

  • For hybrid deployment: multiple instances are deployed on a single machine. For details, see hybrid deployment topology. 
    tiup cluster template --full > topology.yaml
  • For geographically distributed deployment: the TiDB cluster is deployed in geographically distributed data centers. For details, see geographically distributed deployment topology. 
    tiup cluster template --multi-dc > topology.yaml

Run cat topology.yaml to check the contents of the configuration file. Excluding comments, the file looks like this.

global:
  user: "tidb"
  ssh_port: 22
  deploy_dir: "/tidb-deploy"
  data_dir: "/tidb-data"
server_configs: {}
pd_servers:
  - host: 10.0.1.4
  - host: 10.0.1.5
  - host: 10.0.1.6
tidb_servers:
  - host: 10.0.1.7
  - host: 10.0.1.8
  - host: 10.0.1.9
tikv_servers:
  - host: 10.0.1.1
  - host: 10.0.1.2
  - host: 10.0.1.3
monitoring_servers:
  - host: 10.0.1.4
grafana_servers:
  - host: 10.0.1.4
alertmanager_servers:
  - host: 10.0.1.4

Step 4. Run the Deployment Command

Before running the deploy command, use the check and check --apply commands to detect and automatically repair potential cluster risks.

  1. Check potential risks. 
    tiup cluster check ./topology.yaml --user root [-p] [-i /home/root/.ssh/gcp_rsa]
  2. Use automatic repair. 
    tiup cluster check ./topology.yaml --apply --user root [-p] [-i /home/root/.ssh/gcp_rsa]
  3. Deploy the TiDB cluster. 
    tiup cluster deploy tidb-test v6.0.0 ./topology.yaml --user root [-p] [-i /home/root/.ssh/gcp_rsa]

In the tiup cluster deploy command above:

  • tidb-test is the name of the TiDB cluster to deploy.
  • v6.1.0 is the version of the TiDB cluster to deploy. Run tiup list tidb to check the latest supported versions.
  • topology.yaml is the initial configuration file.
  • --user root indicates that cluster deployment is completed by logging in to the target machine as the root user. The root user must have sudo privileges for SSH on the target machine. Alternatively, you can complete deployment with another SSH user that has sudo privileges.
  • [-i] and [-p] are optional. If passwordless login to the target machine is configured, these parameters are not required. Otherwise, choose one of the two parameters. [-i] is the private key of the root user, or another user specified with --user, that can access the target system. [-p] is used to enter the user password interactively. If Deployed cluster tidb-test successfully is displayed at the end of the output log, deployment succeeded.

Step 5. Check Clusters Managed by TiUP

tiup cluster list

TiUP supports managing multiple TiDB clusters. The command above outputs information about all clusters currently managed by TiUP, including cluster name, deployment user, version, and private key information.

Step 6. Check the Status of the Deployed TiDB Cluster

For example, run the following command to check the status of the tidb-test cluster.

tiup cluster display tidb-test

The expected output includes instance ID, role, host, listening port, status, which is Down/inactive because the cluster has not started yet, and directory information.

Step 7. Start the TiDB Cluster

Starting from TiUP Cluster v1.9.0, safe start was introduced as a new startup method. Starting the database this way improves database security. It is recommended to use this method.

After safe start, TiUP automatically generates a password for the TiDB root user and returns the password in the command-line interface.

Method 1: Safe start

tiup cluster start tidb-test --init

If the output is as follows, startup succeeded.

Started cluster `tidb-test` successfully.
The root password of TiDB database has been changed.
The new password is: 'y_+3Hwp=*AWz8971s6'.
Copy and record it to somewhere safe, it is only displayed once, and will not be stored.
The generated password can NOT be got again in future.

Method 2: Standard start

tiup cluster start tidb-test

If the output log includes Started cluster 'tidb-test' successfully, startup succeeded. After standard start, you can log in to the database with the root user without a password.

Step 8. Check the Running Status of the TiDB Cluster

tiup cluster display tidb-test

If the output log shows the Up status, the cluster is working properly.

References