Skip to content

update benchmark in 2.0.x #873

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Dec 4, 2025
Merged

update benchmark in 2.0.x #873

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
85 changes: 41 additions & 44 deletions src/UserGuide/Master/Table/Tools-System/Benchmark.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,50 +40,51 @@ Figure 1-2 *IoT-benchmark Modular Design*

Currently, IoT-benchmark supports the following time series databases, versions and connection methods:

| Database | Version | Connection mmethod |
| :-------------- | :-------------- | :------------------------------------------------------- |
| InfluxDB | v1.x v2.0 | SDK |
| TimescaleDB | -- | JDBC |
| OpenTSDB | -- | HTTP Request |
| QuestDB | v6.0.7 | JDBC |
| TDengine | v2.2.0.2 | JDBC |
| VictoriaMetrics | v1.64.0 | HTTP Request |
| KairosDB | -- | HTTP Request |
| IoTDB | v2.0 v1.x v0.13 | JDBC, SessionByTablet, SessionByRecord, SessionByRecords |
| Database | Version | Connection mmethod |
| :-------------- |:-----------| :------------------------------------------------------- |
| IoTDB | v1.x v2.x | JDBC, SessionByTablet, SessionByRecord, SessionByRecords |
| InfluxDB | v1.x v2.x | SDK |
| TimescaleDB | -- | JDBC |
| OpenTSDB | -- | HTTP Request |
| QuestDB | v6.0.7 | JDBC |
| TDengine | v2.2.0.2 | JDBC |
| VictoriaMetrics | v1.64.0 | HTTP Request |
| KairosDB | -- | HTTP Request |


## 2. **Installation and Operation**

#### **Prerequisites**
### 2.1 **Prerequisites**

1. Java 8
2. Maven 3.6+
3. The corresponding appropriate version of the database, such as Apache IoTDB 2.0

#### **How to Obtain**
### 2.2 **How to Obtain**

- **B****inary package****:** Visit https://github.com/thulab/iot-benchmark/releases to download the installation package. Extract the compressed file into a desired folder for use.

- **Source Code** **Compilation (for** **Apache** **IoTDB 2.0 testing):**
- **Source Code** **Compilation (for** **Apache** **IoTDB 2.0 testing):**

- **Compile the latest IoTDB Session package:** Download the IoTDB source code from https://github.com/apache/iotdb/tree/rc/2.0.1 and run the following command in the root directory to compile the latest IoTDB Session package:
- **Compile the latest IoTDB Session package:** Download the IoTDB source code from https://github.com/apache/iotdb/tree/rc/2.0.5 and run the following command in the root directory to compile the latest IoTDB Session package:

```Bash
mvn clean package install -pl session -am -DskipTests
```
```Bash
mvn clean package install -pl session -am -DskipTests
```

- **Compile the IoT-benchmark test package:** Download the source code from https://github.com/thulab/iot-benchmark and run the following command in the root directory to compile the Apache IoTDB 2.0 test package:.
- **Compile the IoT-benchmark test package:** Download the source code from https://github.com/thulab/iot-benchmark and run the following command in the root directory to compile the Apache IoTDB 2.0 test package:.

```Bash
mvn clean package install -pl iotdb-2.0 -am -DskipTests
```
```Bash
mvn clean package install -pl iotdb-2.0 -am -DskipTests
```

- The compiled test package will be located at:
- The compiled test package will be located at:

```Bash
./iotdb-2.0/target/iotdb-2.0-0.0.1/iotdb-2.0-0.0.1
```
```Bash
./iotdb-2.0/target/iotdb-2.0-0.0.1/iotdb-2.0-0.0.1
```

#### **Test Package Structure**
### 2.3 **Test Package Structure**

The directory structure of the test package is shown below. The test configuration file is `conf/config.properties`, and the test startup scripts are `benchmark.sh` (Linux & MacOS) and `benchmark.bat` (Windows). The detailed usage of the files is shown in the table below.

Expand Down Expand Up @@ -113,7 +114,7 @@ drwxr-xr-x. 2 root root 4096 Jan 10 01:38 lib



#### **Execution** **of** **Tests**
### 2.4 **Execution** **of** **Tests**

1. Modify the configuration file (conf/config.properties) according to test requirements. For example, to test Apache IoTDB 2.0, set the following parameter:

Expand All @@ -127,9 +128,7 @@ drwxr-xr-x. 2 root root 4096 Jan 10 01:38 lib

4. Upon completion, review the results and analyze the test process.

####

#### **Results Interpretation**
### 2.5 **Results Interpretation**

All test log files are stored in the `logs` folder, while test results are saved in the `data/csvOutput` folder. For example, the following result matrix illustrates the test outcome:

Expand All @@ -147,16 +146,14 @@ All test log files are stored in the `logs` folder, while test results are saved

## 3. **Main** **Parameters**

#### IoTDB Service Model
### 3.1 IoTDB Service Model

The `IoTDB_DIALECT_MODE` parameter supports two modes: `tree` and `table`. The default value is `tree`.

- **For IoTDB 2.0 and later versions**, the `IoTDB_DIALECT_MODE` parameter must be specified, and only one mode can be set for each IoTDB instance.
- **IoTDB_DIALECT_MODE = table:**
- The number of devices must be an integer multiple of the number of tables.
- The number of tables must be an integer multiple of the number of databases.
- **IoTDB_DIALECT_MODE = tree:**
- The number of devices must be greater than or equal to the number of databases.

Key Parameters for IoTDB Service Model

Expand All @@ -167,7 +164,7 @@ Key Parameters for IoTDB Service Model
| SENSOR_NUMBER | Integer | `10` | Controls the number of attribute columns in the table model. |
| IoTDB_TABLE_NUMBER | Integer | `1` | Specifies the number of tables when using the table model. |

#### **Working** **M****ode**
### 3.2 **Working** **Mode**

The `BENCHMARK_WORK_MODE` parameter supports four operational modes:

Expand All @@ -185,7 +182,7 @@ Mode configurations are shown in the following below:
| Single database correctness write mode | verificationWriteMode | Writes datasets for correctness verification. | `FILE_PATH` and `DATA_SET` |
| Single database correctness query mode | verificationQueryMode | Queries datasets to verify correctness. | `FILE_PATH` and `DATA_SET` |

#### **Server** **Connection** **Information**
### 3.3 **Server** **Connection** **Information**

Once the working mode is specified, the following parameters must be configured to inform IoT-benchmark of the target time-series database:

Expand All @@ -199,7 +196,7 @@ Once the working mode is specified, the following parameters must be configured
| DB_NAME | String | `test` | Name of the target time-series database. |
| TOKEN | String | - | Authentication token (used for InfluxDB 2.0). |

#### **Write Scenario Parameters**
### 3.4 **Write Scenario Parameters**

| **Parameter** | **Type** | **Example** | D**escription** |
| :------------------------- | :-------------------- | :-------------------------- | :----------------------------------------------------------- |
Expand All @@ -217,7 +214,7 @@ Once the working mode is specified, the following parameters must be configured
| LOOP | Integer | `86400` | Total number of write operations: Each type of operation will be divided according to the proportion defined by `OPERATION_PROPORTION` |
| OPERATION_PROPORTION | Character | `1:0:0:0:0:0:0:0:0:0:0` | Ratio of operation types (write:Q1:Q2:...:Q10). |

#### **Query Scenario Parameters**
### 3.5 **Query Scenario Parameters**

| Parameter | Type | Example | Description |
| :------------------- | :-------- | :---------------------- | :----------------------------------------------------------- |
Expand All @@ -231,7 +228,7 @@ Once the working mode is specified, the following parameters must be configured
| LOOP | Integer | `10` | Total number of query operations: Each type of operation will be divided according to the proportion defined by `OPERATION_PROPORTION` |
| OPERATION_PROPORTION | Character | `0:0:0:0:0:0:0:0:0:0:1` | Ratio of operation types (`write:Q1:Q2:...:Q10`). |

#### **Query Types and Example SQL**
### 3.6 **Query Types and Example SQL**

| Number | Query Type | IoTDB Sample SQL |
| :----- | :----------------------------- | :----------------------------------------------------------- |
Expand All @@ -246,7 +243,7 @@ Once the working mode is specified, the following parameters must be configured
| Q9 | Descending Range Query | `select v1 from root.sg.d1 where time > ? and time < ? order by time desc` |
| Q10 | Descending Range with Filter | `select v1 from root.sg.d1 where time > ? and time < ? and v1 > ? order by time desc` |

#### **Test process and test result persistence**
### 3.7 **Test process and test result persistence**

IoT-benchmark currently supports persisting the test process and test results through configuration parameters.

Expand Down Expand Up @@ -279,9 +276,9 @@ IoT-benchmark currently supports persisting the test process and test results th
2. Stores the test results after test completion.
3. Created if it does not exist.

#### Automation Script
### 3.8 Automation Script

##### One-Click Script Startup
#### One-Click Script Startup

The `cli-benchmark.sh` script allows one-click startup of IoTDB, IoTDB Benchmark monitoring, and IoTDB Benchmark testing. However, please note that this script will clear all existing data in IoTDB during startup, so use it with caution.

Expand All @@ -298,7 +295,7 @@ The `cli-benchmark.sh` script allows one-click startup of IoTDB, IoTDB Benchmark
1. Check test-related logs in the `logs` folder.
2. Check monitoring-related logs in the `server-logs` folder.

##### Automatic Execution of Multiple Tests
#### Automatic Execution of Multiple Tests

Single tests are often insufficient without comparative results. Therefore, IoT-benchmark provides an interface for executing multiple tests in sequence.

Expand Down Expand Up @@ -328,13 +325,13 @@ Then the test process with 3 LOOP parameters of 10, 20, and 50 is executed in se

- Changed parameters persist across subsequent tests unless explicitly reset.

1. **Start the Test:** After configuring the `routine` file, start multi-test execution using the following command
2. **Start the Test:** After configuring the `routine` file, start multi-test execution using the following command

```Bash
> ./rep-benchmark.sh
```

2. Test results will be displayed in the terminal.
Test results will be displayed in the terminal.

**Important Notes:**

Expand Down
Loading