devkuma – TiDB

TiDB Software and Hardware Recommendations

kc@example.com (kc kim) — Fri, 02 Sep 2022 15:38:00 +0900

Network Requirements

To run TiDB, an open source distributed NewSQL database, the following network port configuration is required. Based on the actual TiDB deployment environment, administrators can open the relevant ports on the network and host sides.

Component	Default port	Description
TiDB	4000	Communication port for applications and DBA tools
TiDB	10080	Communication port for reporting TiDB status
TiKV	20160	TiKV communication port
TiKV	20180	Communication port for reporting TiKV status
PD	2379	Communication port between TiDB and PD
PD	2380	Communication port between nodes in the PD cluster
TiFlash	9000	TiFlash TCP service port
TiFlash	8123	TiFlash HTTP service port
TiFlash	3930	TiFlash RAFT and coprocessor service port
TiFlash	20170	TiFlash proxy service port
TiFlash	20292	Port where Prometheus pulls TiFlash Proxy metrics
TiFlash	8234	Port where Prometheus pulls TiFlash metrics
Pump	8250	Pump communication port
Drainer	8249	Drainer communication port
TiCDC	8300	TiCDC communication port
Monitoring	9090	Communication port for the Prometheus service
Monitoring	20120	Communication port for the NgMonitoring service
Node_exporter	9100	Communication port for reporting system information from all TiDB cluster nodes
Blackbox_exporter	9115	Blackbox_exporter communication port. Used to monitor ports in the TiDB cluster.
Grafana	3000	Port for external web monitoring services and client (browser) access
Alertmanager	9093	Alert web service port
Alertmanager	9094	Alert communication port

TiDB Overview

kc@example.com (kc kim) — Mon, 08 Aug 2022 14:32:00 +0900

TiDB(/’taɪdiːbi:/, where “Ti” stands for Titanium) is an open source NewSQL database that supports hybrid transactional and analytical processing (HTAP) workloads. It is compatible with MySQL and provides horizontal scalability, strong consistency, and high availability. TiDB aims to provide users with a one-stop database solution for OLTP (online transaction processing), OLAP (online analytical processing), and HTAP services. TiDB is suitable for a wide range of use cases that require high availability and strong consistency for large-scale data.

Key Features

Horizontal scale-out and simple scale-in

TiDB’s architecture separates computing from storage, so compute or storage capacity can be scaled out or scaled in independently online as needed. The scaling process is transparent to application operations and maintenance teams.
Financial-grade high availability

Data is stored in multiple replicas. Data replicas use the Multi-Raft protocol to replicate transaction logs. A transaction can be committed only after data has been successfully written to most replicas. This ensures strong consistency and availability when a minority of replicas go down. To meet different disaster-resilience requirements, you can configure geographical locations and the number of replicas as needed.
Real-time HTAP

TiDB provides two storage engines. TiKV is a row-based storage engine, and TiFlash is a column-based storage engine. TiFlash uses the Multi-Raft Learner protocol to replicate data from TiKV in real time, ensuring data consistency between the TiKV row-based storage engine and the TiFlash column-based storage engine. TiKV and TiFlash can be deployed on different systems as needed to address HTAP resource isolation.

HTAP (Hybrid Transaction/Analytical Processing)
HTAP (Hybrid Transaction/Analytical Processing) is a term coined by the research firm Gartner. It refers to a next-generation data platform that supports OLTP (Online Transaction Processing) and OLAP (Online Analytical Processing) without data replication. Original article: ["Big Data Frugal Innovation": 8 Convergence Tips from MapR](https://www.ciokorea.com/t/544/12200/30801?page=0,1#csidxd92aa7fca28b93089f3d5aea81955b0)
Cloud-native distributed database

TiDB is a distributed database designed for the cloud, providing flexible scalability, reliability, and security on cloud platforms. Users can scale TiDB flexibly to match changing workload requirements. With TiDB, each piece of data has at least three replicas and can be scheduled across different cloud availability zones to tolerate the loss of an entire data center. TiDB Operator supports TiDB management on Kubernetes and automates tasks related to TiDB cluster operations. This makes it easy to deploy TiDB on clouds that provide managed Kubernetes. TiDB Cloud, a fully managed TiDB service, is the easiest, most cost-effective, and most elastic way to use TiDB in the cloud, allowing you to deploy and run TiDB clusters with just a few clicks.
Compatible with the MySQL 5.7 protocol and MySQL ecosystem

TiDB is compatible with the MySQL 5.7 protocol, common MySQL features, and the MySQL ecosystem. Migrating an application to TiDB often requires no code changes, or only a small amount of code modification. TiDB also provides a set of data migration tools that help migrate application data to TiDB easily.

Use Cases

Financial industry scenarios with high requirements for data consistency, reliability, availability, scalability, and fault tolerance

As many people know, the financial industry has high requirements for data consistency, reliability, availability, scalability, and fault tolerance. Traditional solutions provide services from two data centers in the same city, while a third data center in another city provides disaster recovery but does not serve traffic. This approach has drawbacks such as low resource utilization and high maintenance costs, and its RTO (Recovery Time Objective) and RPO (Recovery Point Objective) do not meet expectations. TiDB uses multiple replicas and the Multi-Raft protocol to schedule data across different data centers, racks, and devices. If some devices fail, the system automatically switches over, enabling RTO ≤ 30 seconds and RPO = 0.
Massive data and high-concurrency scenarios with high requirements for storage capacity, scalability, and concurrency

As applications grow rapidly, data increases sharply. Traditional standalone databases cannot meet data capacity requirements. The solution is to use sharding middleware or a NewSQL database such as TiDB. The latter is cost-effective. TiDB uses a separated compute and storage architecture, so compute or storage capacity can be scaled out or scaled in independently. The compute layer supports up to 512 nodes, each node supports up to 1,000 concurrent executions, and the maximum cluster capacity reaches the PB (petabyte) level.
Real-time HTAP scenarios

With the rapid growth of 5G, the Internet of Things, and artificial intelligence, data generated by enterprises has increased enormously, reaching hundreds of TB (terabytes) or even PB scale. A traditional solution is to use an OLTP database for online transaction applications and use ETL (extract, transform, load) tools to replicate data to an OLAP database for data analysis. This approach has several drawbacks, including high storage costs and low real-time performance. TiDB introduced the TiFlash columnar storage engine in v4.0. Combined with the TiKV row-based storage engine, it turns TiDB into a true HTAP database. With only a small amount of additional storage cost, both online transaction processing and real-time data analysis can be handled in the same system, significantly reducing costs.
Data aggregation and secondary processing scenarios

Most enterprise application data is distributed across various systems. As applications grow, decision-makers need to understand the company’s overall business status and make timely decisions. In this situation, a company needs to aggregate distributed data into the same system and perform secondary processing to generate T+0 or T+1 reports. A traditional solution is to use ETL and Hadoop, but Hadoop systems are complex and have high operations, maintenance, and storage costs. Compared with Hadoop, TiDB is much simpler. You can use ETL or data migration tools provided by TiDB to replicate data into TiDB. Reports can be generated directly using SQL statements.

TiDB Introduction Last modified on 2022-08-11 18:22:49: overview: refine the size of the intro video (#9938) (#9940)

TiDB Experimental Features

kc@example.com (kc kim) — Thu, 01 Sep 2022 18:06:00 +0900

This document introduces experimental features in various versions of TiDB. We recommend that you do not use these features in production environments.

Performance

Support for collecting PREDICATE COLUMNS statistics (introduced in v5.4)
Support for loading statistics synchronously (introduced in v5.4)
Memory quota control for collecting statistics (introduced in v6.1.0)
Randomly samples about 10,000 rows of data to quickly create statistics (introduced in v3.0)

Stability

Improves the stability of indexes selected by the optimizer. It extends statistics capabilities by collecting order-dependent information for multiple columns (introduced in v5.0).
If TiKV is deployed with limited resources and too many read and write requests are processed in the TiKV foreground, CPU resources used to process these requests in the background are occupied, which affects TiKV performance stability. To avoid this situation, use a quota limiter to limit CPU resources used in the foreground. (introduced in v6.0)

Scheduling

Elastic scheduling. This enables a TiDB cluster to dynamically scale out and scale in on Kubernetes based on real-time workloads. This effectively reduces pressure during application peaks and saves overhead. For details, see using TidbCluster autoscaling. (introduced in v4.0)

SQL

Expression indexes. Expression indexes are also called function-based indexes. When creating an index, the indexed field does not have to be a specific column; it can be an expression calculated from one or more columns. This feature helps quickly access calculation-based tables. For details, see expression indexes. (introduced in v4.0)
Generated columns (introduced in v2.1)
User-defined variables (introduced in v2.1)
JSON data types and JSON functions (introduced in v2.1)
Cascades Planner: a top-down query optimizer based on the Cascades framework (introduced in v3.0)
ALTER TABLE ... COMPACT (introduced in v6.1.0)
Changing multiple columns or indexes by using ALTER TABLE (introduced in v5.0.0)

Storage

Titan disabled (introduced in v4.0)
Titan level merge (introduced in v4.0)
Partitions are split into buckets. Buckets are used as units of concurrent queries and improve scan concurrency. (introduced in v6.1.0)
TiKV introduces API V2. (introduced in v6.1.0)

Backup and Restore

RawKV backup and restore (introduced in v3.1)

Data Migration

Use WebUI to manage migration tasks in DM. (introduced in v6.0)

Garbage Collection

Green GC (introduced in v5.0)

Diagnostics

SQL diagnostics (introduced in v4.0)
Cluster diagnostics (introduced in v4.0)

TiDB Experimental Features Last modified on 2022-08-11 18:36:36: Add Cascades Planner to experimental-features (#9854) (#9934)

TiDB Limitations

kc@example.com (kc kim) — Fri, 02 Sep 2022 14:49:00 +0900

This document describes general usage limits in TiDB, including the maximum length of identifiers and the maximum number of supported databases, tables, indexes, partitioned tables, and sequences.

Identifier Length Limits

Identifier type	Maximum length (number of allowed characters)
Database	64
Table	64
Column	64
Index	64
View	64
Sequence	64

Limits on the Total Number of Databases, Tables, Views, and Connections

Identifier type	Maximum number
Databases	Unlimited
Tables	Unlimited
Views	Unlimited
Connections	Unlimited

Single Database Limits

Type	Upper limit
Tables	Unlimited

Single Table Limits

Type	Upper limit (default)
Columns	The default is 1017 and can be adjusted up to 4096.
Indexes	The default is 64 and can be adjusted up to 512.
Rows	Unlimited
Size	Unlimited
Partitions	8192

The upper limit for Columns can be changed with table-column-count-limit.
The upper limit for Indexes can be changed with index-limit.

Single Row Limits

Type	Upper limit
Size	6MB
You can adjust the size limit with the `txn-entry-size-limit` configuration item.

Single Column Limits

Type	Upper limit
Size	6MB

String Type Limits

Type	Upper limit
CHAR	256 characters
BINARY	256 characters
VARBINARY	65535 characters
VARCHAR	16383 characters
TEXT	6MB
BLOB	6MB

SQL Statement Limits

Type	Upper limit
Maximum number of SQL statements in a single transaction	When optimistic transactions and transaction retries are used, the upper limit is 5000.

You can change the limit with the stmt-count-limit configuration item.

Credits Last modified on 2022-04-07 14:42:32: Add TiFlash to the credits.md (#8080)

TiDB

kc@example.com (kc kim) — Wed, 03 Aug 2022 15:06:00 +0900

This page includes translations of parts of the original documentation.

About TiDB

kc@example.com (kc kim) — Thu, 01 Sep 2022 17:59:00 +0900

TiDB Installation

kc@example.com (kc kim) — Wed, 03 Aug 2022 15:06:00 +0900

Quick Start

kc@example.com (kc kim) — Fri, 02 Sep 2022 15:18:00 +0900

TiDB Operator - Automatic Failover

kc@example.com (kc kim) — Tue, 01 Nov 2022 01:25:00 +0900

What Is Automatic Failover?

If a node in a TiDB cluster fails and stops working for some reason, TiDB appears to automatically add a new node and preserve cluster availability.

Automatic Failover Test

For this test, I will kill one of two TiDB server processes and check whether it recovers automatically.

First, check the tidb-server processes.

]$ ps -ef | grep tidb-server | grep -v grep
kimkc    14506 14429  0 11:53 pts/0    00:00:22 /home/kimkc/.tiup/components/tidb/v6.1.0/tidb-server -P 4000 --store=tikv --host=127.0.0.1 --status=10080 --path=127.0.0.1:2379,127.0.0.1:2382,127.0.0.1:2384 --log-file=/home/kimkc/.tiup/data/TLr4XQE/tidb-0/tidb.log
kimkc    14508 14429  0 11:53 pts/0    00:00:26 /home/kimkc/.tiup/components/tidb/v6.1.0/tidb-server -P 4001 --store=tikv --host=127.0.0.1 --status=10081 --path=127.0.0.1:2379,127.0.0.1:2382,127.0.0.1:2384 --log-file=/home/kimkc/.tiup/data/TLr4XQE/tidb-1/tidb.log

Forcefully terminate the process using port 4001.

$ kill -9 14508
[kimkc@localhost ~]$ ps -ef | grep tidb-server | grep -v grep
kimkc    14506 14429  0 11:53 pts/0    00:00:22 /home/kimkc/.tiup/components/tidb/v6.1.0/tidb-server -P 4000 --store=tikv --host=127.0.0.1 --status=10080 --path=127.0.0.1:2379,127.0.0.1:2382,127.0.0.1:2384 --log-file=/home/kimkc/.tiup/data/TLr4XQE/tidb-0/tidb.log
[kimkc@localhost ~]$

You can see that one TiDB server process has died and only one remains. Wait for a while.

Even after waiting for some time, it does not recover.

[kimkc@localhost ~]$ ps -ef | grep tidb-server | grep -v grep
kimkc    14506 14429  0 11:53 pts/0    00:00:27 /home/kimkc/.tiup/components/tidb/v6.1.0/tidb-server -P 4000 --store=tikv --host=127.0.0.1 --status=10080 --path=127.0.0.1:2379,127.0.0.1:2382,127.0.0.1:2384 --log-file=/home/kimkc/.tiup/data/TLr4XQE/tidb-0/tidb.log
[kimkc@localhost ~]$

This shows that recovery does not happen with the basic functionality alone.

TiDB Operator

Automatic failover additionally requires a separate tool called TiDB Operator.

It appears to be a tool for managing TiDB clusters in a Kubernetes cluster and automating related tasks. With this, TiDB can become truly cloud native.

Cluster Architecture

kc@example.com (kc kim) — Thu, 01 Sep 2022 17:59:00 +0900

TiDB Automatic Failover

kc@example.com (kc kim) — Tue, 01 Nov 2022 01:25:00 +0900

One of TiDB’s major strengths is high availability.

High availability is the property that enables information systems such as servers, networks, and programs to operate normally and continuously for a considerably long period of time. High availability means that “availability is high”; it does not mean that a system will “never fail.”