Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
54 changes: 52 additions & 2 deletions br/br-log-architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,35 @@ summary: 了解 TiDB 的日志备份与 PITR 的架构设计。

日志备份的流程如下:

![BR log backup process design](/media/br/br-log-backup-ts.png)
```mermaid
sequenceDiagram
actor User
participant BR
participant PD
participant TiKV
participant TiDB
participant Storage

User->>BR: Run `br log start`
BR->>PD: Register log backup task
TiKV->>PD: Fetch log backup task
par TiKV handles the local log backup task
loop
TiKV->>TiKV: Read KV change data
TiKV->>PD: Fetch global checkpoint ts
TiKV->>TiKV: Generate local metadata
TiKV->>Storage: Upload log data & metadata
TiKV->>PD: Configure GC
end
and
loop
TiDB->>TiKV: Watch backup progress
TiDB->>PD: Report global checkpoint ts
end
end
User->>BR: Run `br log status`
BR->>PD: Fetch status of log backup task
```

系统组件和关键概念:

Expand Down Expand Up @@ -53,7 +81,29 @@ summary: 了解 TiDB 的日志备份与 PITR 的架构设计。

PITR 的流程如下:

![Point-in-time recovery process design](/media/br/pitr-ts.png)
```mermaid
sequenceDiagram
actor User
participant BR
participant TiKV
participant PD
participant Storage

User->>BR: Run `br restore point`
BR->>TiKV: Restore full data
loop restore log data
BR->>Storage: Read backup data
BR->>PD: Fetch Region info
BR->>TiKV: Request TiKV to restore data
loop TiKV handles restore request
TiKV->>Storage: Download KVs
TiKV->>TiKV: Rewrite KVs
TiKV->>TiKV: Apply KVs
end
TiKV->>BR: Report restore result
BR->>BR: Handle all restore results
end
```

完整的 PITR 交互流程描述如下:

Expand Down
45 changes: 43 additions & 2 deletions br/br-snapshot-architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,28 @@ summary: 了解 TiDB 快照备份与恢复功能的架构设计。

集群快照数据备份的流程如下:

![snapshot backup process design](/media/br/br-snapshot-backup-ts.png)
```mermaid
sequenceDiagram
actor User
participant BR
participant PD
participant TiKV
participant Storage

User->>BR: Run `br backup full`
BR->>PD: Pause GC
BR->>PD: Fetch TiKV and Region info
BR->>TiKV: Request TiKV to back up data
loop TiKV handles the local snapshot backup task
TiKV->>TiKV: Scan KVs
TiKV->>TiKV: Generate SST
TiKV->>Storage: Upload SST
end
TiKV->>BR: Report backup result
BR->>BR: Handle all backup results
BR->>TiKV: Back up schemas
BR->>Storage: Upload backup metadata
```

完整的备份交互流程描述如下:

Expand Down Expand Up @@ -49,7 +70,27 @@ summary: 了解 TiDB 快照备份与恢复功能的架构设计。

恢复集群快照备份数据的流程如下:

![snapshot restore process design](/media/br/br-snapshot-restore-ts.png)
```mermaid
sequenceDiagram
actor User
participant BR
participant PD
participant TiKV
participant Storage

User->>BR: Run `br restore`
BR->>PD: Pause Region schedule
BR->>TiKV: Restore schema
BR->>PD: Split and scatter Regions
BR->>TiKV: Request TiKV to restore data
loop TiKV handles restore request
TiKV->>Storage: Download SST
TiKV->>TiKV: Rewrite KVs
TiKV->>TiKV: Ingest SST
end
TiKV->>BR: Report restore result
BR->>BR: Handle all restore results
```

完整的恢复交互流程描述如下:

Expand Down
30 changes: 29 additions & 1 deletion dm/feature-shard-merge-pessimistic.md
Original file line number Diff line number Diff line change
Expand Up @@ -78,7 +78,35 @@ DM 在悲观模式下进行分表 DDL 的迁移有以下几点使用限制:

基于上述例子,本部分介绍了 DM 在默认的悲观模式下合库合表过程中进行 DDL 迁移的实现原理。

![shard-ddl-flow](/media/dm/shard-ddl-flow.png)
```mermaid
---
config:
themeCSS: |
/* hide the ugly borders */
rect.rect {
stroke: none;
}
---
sequenceDiagram
autonumber
box rgba(0,255,0,0.08)
participant Worker1 as DM-worker 1
end
box rgba(255,255,0,0.08)
participant Master as DM-master
end
box rgba(0,255,0,0.08)
participant Worker2 as DM-worker 2
end

Worker1->>Master: 1. DDL info
Master->>Worker1: 2. DDL lock info
Worker2->>Master: 3. DDL info
Master->>Worker2: 4. DDL lock info
Master->>Worker1: 5. DDL execute request
Worker1->>Master: 6. DDL executed
Master-->>Worker2: 7. DDL ignore request
```

在这个例子中,DM-worker-1 负责迁移来自 MySQL 实例 1 的数据,DM-worker-2 负责迁移来自 MySQL 实例 2 的数据,DM-master 负责协调多个 DM-worker 间的 DDL 迁移。

Expand Down
101 changes: 98 additions & 3 deletions pessimistic-transaction.md
Original file line number Diff line number Diff line change
Expand Up @@ -142,15 +142,92 @@ TiDB 在悲观事务模式下支持了 2 种隔离级别:

TiDB 悲观锁复用了乐观锁的两阶段提交逻辑,重点在 DML 执行时做了改造。

![TiDB 悲观事务的提交流程](/media/pessimistic-transaction-commit.png)
```mermaid
---
config:
themeCSS: |
/* workaround for https://github.com/mermaid-js/mermaid/issues/523 */
/* mark the two "Lock" arrows as red, by restyling the dashed arrows */
line.messageLine1 {
stroke: #d32f2f;
stroke-dasharray: none !important;
}
/* make sure the arrow heads inherit the stroke color (another bug in mermaid) */
#arrowhead path {
fill: context-stroke;
stroke: context-stroke;
}
---
sequenceDiagram
participant Client
participant TiDB
participant TiKV

Client->>TiDB: BEGIN
rect rgba(255, 0, 0, 0.08)
Client->>TiDB: DML
TiDB-->>TiKV: Lock
TiDB-->>TiKV: Lock
end
rect rgba(0, 0, 0, 0.04)
Client->>TiDB: COMMIT
TiDB->>TiKV: Prewrite
TiDB->>TiKV: Commit
end
```

在两阶段提交之前增加了 Acquire Pessimistic Lock 阶段,简要步骤如下。

1. (同乐观锁)TiDB 收到来自客户端的 begin 请求,获取当前时间戳作为本事务的 StartTS。
2. TiDB 收到来自客户端的更新数据的请求:TiDB 向 TiKV 发起加悲观锁请求,该锁持久化到 TiKV。
3. (同乐观锁)客户端发起 commit,TiDB 开始执行与乐观锁一样的两阶段提交。

![TiDB 中的悲观事务](/media/pessimistic-transaction-in-tidb.png)
```mermaid
---
title: Pessimistic Transaction in TiDB
---
sequenceDiagram
participant client
participant TiDB
participant PD
participant TiKV

client->>TiDB: begin
TiDB->>PD: get ts as start_ts
loop execute SQL
rect rgba(0, 0, 0, 0.04)
alt do read
TiDB->>TiKV: get data from TiKV with start_ts
TiDB->>client: return read result
else do write
rect rgba(255, 0, 0, 0.08)
loop if has write conflict
TiDB->>PD: get ts as for_update_ts
TiDB->>TiDB: write in cache
TiDB->>TiKV: acquire pessimistic locks parallelly
end
end
TiDB->>client: return write result
end
end
end
client->>TiDB: commit
opt start 2PC
rect rgba(0, 0, 0, 0.04)
par prewrite
TiDB->>TiKV: prewrite each key in cache with start_ts parallelly
end
end
rect rgba(0, 0, 0, 0.04)
par commit
TiDB->>PD: get ts as commit_ts
TiDB->>TiKV: commit primary_key with commit_ts first
TiDB->>client: success
TiDB->>TiKV: commit each secondary_key with commit_ts parallelly
end
end
end
```

相关细节本节不再赘述,详情可阅读 [TiDB 悲观锁实现原理](https://pingkai.cn/tidbcommunity/blog/7730ed79)。

Expand All @@ -164,7 +241,25 @@ TiDB 悲观锁复用了乐观锁的两阶段提交逻辑,重点在 DML 执行

如果业务逻辑依赖加锁或等锁机制,或者即使在集群异常情况下也要尽可能保证事务提交的成功率,应关闭 pipelined 加锁功能。

![Pipelined pessimistic lock](/media/pessimistic-transaction-pipelining.png)
```mermaid
---
title: pipelined pessimistic lock
---
sequenceDiagram
participant Client
participant TiDB
participant TiKV1
participant TiKV2
participant TiKV3

loop
Client->>TiDB: DML
TiDB->>TiKV1: Acquire pessimistic locks
TiKV1->>TiDB: OK
TiKV1--)TiKV2: Log replication
TiKV1--)TiKV3: Log replication
end
```

该功能默认开启,可修改 TiKV 配置关闭:

Expand Down
Loading