Skip to content

Commit e638b13

Browse files
tarofaceflorence-crl
taroface
and
florence-crl
authored
Molt updates (#19675)
* remove CRDB from supported source DBs * clarify replicator metrics endpoint --------- Co-authored-by: Florence Morris <[email protected]>
1 parent 681e75e commit e638b13

File tree

6 files changed

+20
-12
lines changed

6 files changed

+20
-12
lines changed

‎src/current/_includes/molt/migration-stop-replication.md

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,13 @@
22

33
1. Wait for replication to drain, which means that all transactions that occurred on the source database have been fully processed and replicated to CockroachDB. There are two ways to determine that replication has fully drained:
44
- When replication is caught up, you will not see new `upserted rows` logs.
5-
- If you set up the replication metrics endpoint with `--metricsAddr` in the preceding steps, use the following Prometheus alert expression to observe when the combined rate of upserts and deletes is `0` for each schema:
5+
- If you set up the replication metrics endpoint with `--metricsAddr` in the preceding steps, metrics are available at:
6+
7+
~~~
8+
http://{host}:{port}/_/varz
9+
~~~
10+
11+
Use the following Prometheus alert expression to observe when the combined rate of upserts and deletes is `0` for each schema:
612

713
~~~
814
sum by (schema) (rate(apply_upserts_total[$__rate_interval]) + rate(apply_deletes_total[$__rate_interval]))

‎src/current/_includes/molt/replicator-flags.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -11,14 +11,14 @@ The following flags are set with [`--replicator-flags`](#global-flags) and can b
1111
| `--gracePeriod` | `DURATION` | Allow background processes to exit.<br><br>**Default:** `30s` |
1212
| `--logDestination` | `STRING` | Write logs to a file. If not specified, write logs to `stdout`. |
1313
| `--logFormat` | `STRING` | Choose log output format: `"fluent"`, `"text"`.<br><br>**Default:** `"text"` |
14-
| `--metricsAddr` | `STRING` | A `host:port` on which to serve metrics and diagnostics. |
14+
| `--metricsAddr` | `STRING` | A `host:port` on which to serve metrics and diagnostics. The metrics endpoint is `http://{host}:{port}/_/varz`. |
1515
| `--parallelism` | `INT` | The number of concurrent database transactions to use.<br><br>**Default:** `16` |
1616
| `--quiescentPeriod` | `DURATION` | How often to retry deferred mutations.<br><br>**Default:** `10s` |
1717
| `--retireOffset` | `DURATION` | How long to delay removal of applied mutations.<br><br>**Default:** `24h0m0s` |
1818
| `--scanSize` | `INT` | The number of rows to retrieve from the staging database used to store metadata for [replication modes](#fetch-mode).<br><br>**Default:** `10000` |
1919
| `--schemaRefresh` | `DURATION` | How often a watcher will refresh its schema. If this value is zero or negative, refresh behavior will be disabled.<br><br>**Default:** `1m0s` |
2020
| `--sourceConn` | `STRING` | The source database's connection string. |
21-
| `--stageDisableCreateTableReaderIndex` | `BOOL` | Disable the creation of partial covering indexes to improve read performance on staging tables. Set to `true` if creating indexes on existing tables would cause a significant operational impact.<br><br>**Default:** `false` |
21+
| `--stageDisableCreateTableReaderIndex` | `BOOL` | Disable the creation of partial covering indexes to improve read performance on staging tables. Set to `true` if creating indexes on existing tables would cause a significant operational impact.<br><br>**Default:** `false` |
2222
| `--stageMarkAppliedLimit` | `INT` | Limit the number of mutations to be marked applied in a single statement.<br><br>**Default:** `100000` |
2323
| `--stageSanityCheckPeriod` | `DURATION` | How often to validate staging table apply order (`-1` to disable).<br><br>**Default:** `10m0s` |
2424
| `--stageSanityCheckWindow` | `DURATION` | How far back to look when validating staging table apply order.<br><br>**Default:** `1h0m0s` |

‎src/current/molt/migrate-in-phases.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -113,7 +113,7 @@ The following example specifies that the `employees` table should be watched for
113113
--non-interactive \
114114
--mode replication-only \
115115
--pglogical-replication-slot-name cdc_slot \
116-
--replicator-flags '--metricsAddr: 30005'
116+
--replicator-flags '--metricsAddr :30005'
117117
~~~
118118
</section>
119119

@@ -128,12 +128,12 @@ The following example specifies that the `employees` table should be watched for
128128
--table-filter 'employees' \
129129
--non-interactive \
130130
--mode replication-only \
131-
--replicator-flags '--defaultGTIDSet 4c658ae6-e8ad-11ef-8449-0242ac140006:1-29 --metricsAddr: 30005'
131+
--replicator-flags '--defaultGTIDSet 4c658ae6-e8ad-11ef-8449-0242ac140006:1-29 --metricsAddr :30005'
132132
~~~
133133
</section>
134134

135135
{{site.data.alerts.callout_info}}
136-
`--metricsAddr` enables a Prometheus-compatible metrics endpoint (e.g., on port `30005`) where replication metrics will be served.
136+
`--metricsAddr` enables a Prometheus-compatible metrics endpoint at `http://{host}:{port}/_/varz` where replication metrics will be served. In this example, the endpoint is `http://localhost:30005/_/varz`.
137137
{{site.data.alerts.end}}
138138

139139
{% include molt/fetch-replication-output.md %}

‎src/current/molt/migrate-to-cockroachdb.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -64,7 +64,7 @@ Start the initial load of data into the target database. Continuous replication
6464
--non-interactive \
6565
--mode data-load-and-replication \
6666
--pglogical-replication-slot-name cdc_slot \
67-
--replicator-flags '--metricsAddr: 30005'
67+
--replicator-flags '--metricsAddr :30005'
6868
~~~
6969
</section>
7070

@@ -79,12 +79,12 @@ Start the initial load of data into the target database. Continuous replication
7979
--table-handling truncate-if-exists \
8080
--non-interactive \
8181
--mode data-load-and-replication \
82-
--replicator-flags '--metricsAddr: 30005'
82+
--replicator-flags '--metricsAddr :30005'
8383
~~~
8484
</section>
8585

8686
{{site.data.alerts.callout_info}}
87-
`--metricsAddr` enables a Prometheus-compatible metrics endpoint (e.g., on port `30005`) where replication metrics will be served.
87+
`--metricsAddr` enables a Prometheus-compatible metrics endpoint at `http://{host}:{port}/_/varz` where replication metrics will be served. In this example, the endpoint is `http://localhost:30005/_/varz`.
8888
{{site.data.alerts.end}}
8989

9090
{% include molt/fetch-data-load-output.md %}

‎src/current/molt/molt-fetch.md

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -15,7 +15,6 @@ The following source databases are currently supported:
1515

1616
- PostgreSQL 11-16
1717
- MySQL 5.7, 8.0 and later
18-
- CockroachDB
1918

2019
## Installation
2120

@@ -900,6 +899,10 @@ You can use the `cdc_cursor` value with an external change data capture (CDC) to
900899

901900
By default, MOLT Fetch exports [Prometheus](https://prometheus.io/) metrics at `127.0.0.1:3030/metrics`. You can configure this endpoint with the `--metrics-listen-addr` [flag](#global-flags).
902901

902+
{{site.data.alerts.callout_info}}
903+
If [replication](#fetch-mode) is active, metrics from the `replicator` process are enabled by setting the `--metricsAddr` [replication flag](#replication-flags), and are served at `http://host:port/_/varz`.
904+
{{site.data.alerts.end}}
905+
903906
Cockroach Labs recommends monitoring the following metrics:
904907

905908
| Metric Name | Description |

‎src/current/molt/molt-verify.md

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -23,11 +23,10 @@ For a demo of MOLT Verify, watch the following video:
2323

2424
## Supported databases
2525

26-
The following databases are currently supported:
26+
The following source databases are currently supported:
2727

2828
- PostgreSQL 12-16
2929
- MySQL 5.7, 8.0 and later
30-
- CockroachDB
3130

3231
## Installation
3332

0 commit comments

Comments
 (0)