You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: tiflash/troubleshoot-tiflash.md
+9-6
Original file line number
Diff line number
Diff line change
@@ -153,9 +153,9 @@ If TiFlash replicas consistently fail to be created since the TiDB cluster is de
153
153
- If `true` is returned, go to the next step.
154
154
- If `false` is returned, [enable the Placement Rules feature](/configure-placement-rules.md#enable-placement-rules) and go to the next step.
155
155
156
-
2. Check whether the TiFlash process is working normally by the `UpTime` on the TiFlash-Summary monitoring panel.
156
+
2. Check whether the TiFlash process is working normally by the `UpTime` on the TiFlash-Summary Grafana panel.
157
157
158
-
3. Check whether the connection between TiFlash and PD is normal through `pd-ctl`.
158
+
3. Check whether the connection between TiFlash and PD is normal.
159
159
160
160
```shell
161
161
tiup ctl:nightly pd -u http://${pd-ip}:${pd-port} store
@@ -205,14 +205,15 @@ If none of the above configurations or TiFlash status show abnormalities, please
205
205
206
206
## Data is not replicated to TiFlash
207
207
208
-
After deploying a TiFlash node and starting replication (by performing the ALTER operation), no data is replicated to it. In this case, you can identify and address the problem by following the steps below:
208
+
After deploying a TiFlash node and starting replication by executing `ALTER TABLE ... SET TIFLASH REPLICA ...`, no data is replicated to it. In this case, you can identify and address the problem by following the steps below:
209
209
210
210
1. Check whether the replication is successful by running the `ALTER table <tbl_name>set tiflash replica <num>`command and check the output.
211
211
212
212
- If there is output, go to the next step.
213
213
- If there is no output, run the `SELECT * FROM information_schema.tiflash_replica`command to check whether TiFlash replicas have been created. If not, run the `ALTER table ${tbl_name}set tiflash replica ${num}`command again
214
214
- Check whether the DDL statement is executed as expected through [ADMIN SHOW DDL](/sql-statements/sql-statement-admin-show-ddl.md). Or there are any other DDL statement that block altering TiFlash replica statement being executed.
215
215
- Check whether any DML statement is executed on the same table through [SHOW PROCESSLIST](/sql-statements/sql-statement-show-processlist.md) that blocks altering TiFlash replica statement being executed.
216
+
- If nothing is blocking the `ALTER TABLE ... SET TIFLASH REPLICA ...` being executed, go to the next step.
216
217
217
218
2. Check whether TiFlash Region replication runs correctly.
218
219
@@ -221,7 +222,7 @@ After deploying a TiFlash node and starting replication (by performing the ALTER
221
222
- If changes are detected, it indicates TiFlash replication is functioning normally (though potentially at a slower pace). Please refer to the "Data replication is slow" section for optimization configurations.
222
223
- If no, TiFlash replication is abnormal. In `tidb.log`, search the log saying `Tiflash replica is not available`. Check whether `progress` of the corresponding table is updated. If not, go to the next step.
223
224
224
-
3. Check whether TiDB has created any placement rule fortables.
225
+
3. Check whether TiDB has created any placement rule forthe table.
225
226
226
227
Search the logs of TiDB DDL Owner and check whether TiDB has notified PD to add placement rules. For non-partitioned tables, search `ConfigureTiFlashPDForTable`. For partitioned tables, search `ConfigureTiFlashPDForPartitions`.
227
228
@@ -230,11 +231,13 @@ After deploying a TiFlash node and starting replication (by performing the ALTER
230
231
231
232
4. Check whether PD has configured any placement rule for tables.
232
233
233
-
Run the `curl http://<pd-ip>:<pd-port>/pd/api/v1/config/rules/group/tiflash`command to view all TiFlash placement rules on the current PD. If a rule with the ID being `table-<table_id>-r` is found, the PD has configured a placement rule successfully.
234
+
Run the `curl http://<pd-ip>:<pd-port>/pd/api/v1/config/rules/group/tiflash`command to view all TiFlash placement rules on the current PD.
235
+
- If a rule with the ID being `table-<table_id>-r` is found, the PD has configured a placement rule successfully, go to the next step.
236
+
- If not, collect logs of the corresponding component for troubleshooting.
234
237
235
238
5. Check whether the PD schedules properly.
236
239
237
-
Search the `pd.log` file for the `table-<table_id>-r` keyword and scheduling behaviors like `add operator`.
240
+
Search the `pd.log` file for the `table-<table_id>-r` keyword and scheduling behaviors like `add operator`. Or check whether there are `add-rule-peer` operator on the "Operator/Schedule operator create" of PD Dashboard on Grafana.
238
241
239
242
- If the keyword is found, the PD schedules properly.
0 commit comments