-
Notifications
You must be signed in to change notification settings - Fork 1.2k
*: add TiDB workload repository #20127
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from 7 commits
Commits
Show all changes
12 commits
Select commit
Hold shift + click to select a range
2258f3c
initial trans
hfxsd bb7427b
Update system-variable-reference.md
hfxsd 134a2a9
Update workloadrepo.md
hfxsd a34924e
Update TOC.md
hfxsd e3a084c
Update system-variable-reference.md
hfxsd c278b89
Apply suggestions from code review
hfxsd e4709d1
Apply suggestions from code review
hfxsd 4760bb5
Update workloadrepo.md
hfxsd 3a8c461
Apply suggestions from code review
hfxsd a5c658a
sync en commit
hfxsd a8bf2b8
Update sql-statement-admin.md
hfxsd 89ce68a
Update sql-statement-admin.md
hfxsd File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,117 @@ | ||
--- | ||
hfxsd marked this conversation as resolved.
Show resolved
Hide resolved
|
||
title: TiDB Workload Repository | ||
summary: 介绍用于收集和存储 TiDB 集群历史工作负载数据的 Workload Repository 系统。 | ||
--- | ||
|
||
# TiDB Workload Repository | ||
|
||
Workload Repository 是一个用于收集和存储 TiDB 集群历史工作负载数据的系统。它会定期采样各种系统表,以跟踪集群的性能和使用模式。 | ||
hfxsd marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
hfxsd marked this conversation as resolved.
Show resolved
Hide resolved
|
||
## 启用 Workload Repository | ||
|
||
要启用 Workload Repository,可以设置 [`tidb_workload_repository_dest`](/system-variables.md#tidb_workload_repository_dest-从-v900-版本开始引入) 系统变量: | ||
|
||
```sql | ||
SET GLOBAL tidb_workload_repository_dest = 'table'; | ||
``` | ||
|
||
要禁用它,执行以下语句: | ||
|
||
```sql | ||
SET GLOBAL tidb_workload_repository_dest = ''; | ||
``` | ||
|
||
## 数据收集 | ||
|
||
Workload Repository 将数据存储在 `WORKLOAD_SCHEMA` 数据库下的表中。它通过两种不同的方法收集数据: | ||
|
||
- 快照采样过程,按可配置的时间间隔运行,通常为每小时一次,也可以手动触发。 | ||
hfxsd marked this conversation as resolved.
Show resolved
Hide resolved
|
||
- 基于时间的采样过程,按较短的时间间隔运行,通常为每 5 秒一次。 | ||
|
||
## 快照采样过程(默认每小时) | ||
|
||
快照采样过程每 15 分钟到 2 小时运行一次,从存储累计指标的内存表中采样数据。快照由指定时间间隔内的某个 TiDB 节点发起,过程如下: | ||
hfxsd marked this conversation as resolved.
Show resolved
Hide resolved
hfxsd marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
hfxsd marked this conversation as resolved.
Show resolved
Hide resolved
|
||
1. 从发起节点向 `HIST_SNAPSHOTS` 表插入一行,记录快照 ID、开始和结束时间戳以及服务器版本信息。 | ||
hfxsd marked this conversation as resolved.
Show resolved
Hide resolved
|
||
2. 在每个 TiDB 节点上,将源表中的所有行复制到带有 `HIST_` 前缀的对应历史表中。复制的数据包括源表中的原始列以及用于时间戳、实例 ID 和快照 ID 的附加列。 | ||
hfxsd marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
注意,采样表返回的数据内容与查询这些数据的 TiDB 节点相关。 | ||
|
||
从以下表中采样数据: | ||
|
||
| 表名 | 目标表 | 描述 | | ||
hfxsd marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| --- | --- | --- | | ||
| `TIDB_INDEX_USAGE` | `HIST_TIDB_INDEX_USAGE` | 索引使用统计信息 | | ||
| `TIDB_STATEMENTS_STATS` | `HIST_TIDB_STATEMENTS_STATS` | 语句统计信息 | | ||
| `CLIENT_ERRORS_SUMMARY_BY_HOST` | `HIST_CLIENT_ERRORS_SUMMARY_BY_HOST` | 按主机的客户端错误摘要 | | ||
| `CLIENT_ERRORS_SUMMARY_BY_USER` | `HIST_CLIENT_ERRORS_SUMMARY_BY_USER` | 按用户的客户端错误摘要 | | ||
| `CLIENT_ERRORS_SUMMARY_GLOBAL` | `HIST_CLIENT_ERRORS_SUMMARY_GLOBAL` | 全局客户端错误摘要 | | ||
hfxsd marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
可以通过 [`tidb_workload_repository_snapshot_interval`](/system-variables.md#tidb_workload_repository_snapshot_interval-从-v900-版本开始引入) 控制快照采样间隔: | ||
|
||
```sql | ||
SET GLOBAL tidb_workload_repository_snapshot_interval = 900; -- 将间隔设置为 15 分钟 | ||
``` | ||
|
||
## 手动快照 | ||
|
||
请注意,虽然快照采样过程会根据配置的时间间隔自动运行,但你也可以使用以下 SQL 语句触发手动快照: | ||
|
||
hfxsd marked this conversation as resolved.
Show resolved
Hide resolved
|
||
```sql | ||
ADMIN WORKLOAD REPOSITORY TAKE SNAPSHOT; | ||
``` | ||
|
||
hfxsd marked this conversation as resolved.
Show resolved
Hide resolved
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 运行这个语句是否有权限要求,如果有,这里需要补充下 |
||
手动快照并不会改变下一次自动快照的发生时间。 | ||
hfxsd marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
## 基于时间的采样过程(默认每 5 秒) | ||
|
||
基于时间的采样间隔可设置为 1 秒到 600 秒之间的任意时间,从各个记录瞬时状态的系统表中采样数据。 | ||
|
||
hfxsd marked this conversation as resolved.
Show resolved
Hide resolved
|
||
当基于时间的采样过程运行时,源表中的所有行都会被复制到带有 `HIST_` 前缀的对应历史表中。复制的数据包括源表中的原始列以及用于时间戳和实例 ID 的附加列。 | ||
hfxsd marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
与快照采样过程不同,不会向 `HIST_SNAPSHOTS` 表添加行。 | ||
|
||
注意,采样表返回的数据内容与查询这些数据的 TiDB 节点相关。 | ||
|
||
从以下表中采样数据: | ||
|
||
| 表名 | 目标表 | 描述 | | ||
hfxsd marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| --- | --- | --- | | ||
| `PROCESSLIST` | `HIST_PROCESSLIST` | 活跃会话 | | ||
| `DATA_LOCK_WAITS` | `HIST_DATA_LOCK_WAITS` | 数据锁等待 | | ||
| `TIDB_TRX` | `HIST_TIDB_TRX` | 活跃事务 | | ||
| `MEMORY_USAGE` | `HIST_MEMORY_USAGE` | 内存使用情况 | | ||
| `DEADLOCKS` | `HIST_DEADLOCKS` | 死锁信息 | | ||
hfxsd marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
可以通过 [`tidb_workload_repository_active_sampling_interval`](/system-variables.md#tidb_workload_repository_active_sampling_interval-从-v900-版本开始引入) 控制基于时间的采样间隔: | ||
|
||
```sql | ||
SET GLOBAL tidb_workload_repository_active_sampling_interval = 20; -- 将间隔设置为 20 秒 | ||
``` | ||
|
||
hfxsd marked this conversation as resolved.
Show resolved
Hide resolved
|
||
把间隔设置为 `0` 将会禁用基于时间的采样。 | ||
hfxsd marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
## 数据保留 | ||
|
||
系统会根据保留期设置自动清除数据,并使用分区进行高效的数据管理。 | ||
hfxsd marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
[`tidb_workload_repository_retention_days`](/system-variables.md#tidb_workload_repository_retention_days-从-v900-版本开始引入) 变量控制 Workload Repository 中历史数据的保留期。例如,要保留 30 天的数据,运行以下命令: | ||
hfxsd marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
```sql | ||
SET GLOBAL tidb_workload_repository_retention_days = 30; | ||
``` | ||
|
||
此变量的较高值允许更长的数据保留时间,这可能有助于工作负载分析,但会增加存储需求。 | ||
hfxsd marked this conversation as resolved.
Show resolved
Hide resolved
hfxsd marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
## 注意事项 | ||
|
||
- 启用 Workload Repository 可能会对系统性能产生轻微影响。 | ||
hfxsd marked this conversation as resolved.
Show resolved
Hide resolved
|
||
- 采样间隔设置得过低可能会增加系统开销。 | ||
hfxsd marked this conversation as resolved.
Show resolved
Hide resolved
hfxsd marked this conversation as resolved.
Show resolved
Hide resolved
|
||
- 将保留天数设置为 `0` 会禁用旧数据的自动清除。 | ||
|
||
## 最佳实践 | ||
|
||
- 从默认设置开始,根据你的监控需求进行调整。 | ||
- 根据存储容量设置合理的保留期。 | ||
- 监控 `WORKLOAD_SCHEMA` 数据库的大小。 | ||
- 在生产环境中使用较长的采样间隔以最小化开销。 |
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.