Skip to content

Conversation

@Caideyipi
Copy link
Collaborator

Description

As the title said.


This PR has:

  • been self-reviewed.
    • concurrent read
    • concurrent write
    • concurrent read and write
  • added documentation for new or modified features or behaviors.
  • added Javadocs for most classes and all non-trivial methods.
  • added or updated version, license, or notice information
  • added comments explaining the "why" and the intent of the code wherever would not be obvious
    for an unfamiliar reader.
  • added unit tests or modified existing tests to cover new code paths, ensuring the threshold
    for code coverage.
  • added integration tests.
  • been tested in a test IoTDB cluster.

Key changed/added classes (or packages if there are too many classes) in this PR

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR adds validation for OBJECT type fields in table schemas to prevent invalid characters in device IDs, table names, and object names. The validation ensures that these identifiers do not equal '.', '..', or contain './' or '.\', which would cause issues with the file system storage backend.

Key changes:

  • Adds needCheck4Object flag to track tables with OBJECT columns and validation methods to check for invalid patterns
  • Implements RPC endpoint checkDeviceIdForObject to validate existing device IDs when OBJECT columns are added to tables
  • Renames schema lock type from VALIDATE_VS_DELETION_TABLE to AVOID_CONCURRENT_DEVICE_ALTER_TABLE for clarity

Reviewed Changes

Copilot reviewed 21 out of 21 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
datanode.thrift Adds TCheckDeviceIdForObjectReq struct and checkDeviceIdForObject RPC method
TsTable.java Adds needCheck4Object flag and validation methods for OBJECT field restrictions
DataNodeTableCache.java Sets needCheck4Object flag during initialization, pre-update, and rollback operations
MTreeBelowSGMemoryImpl.java Renames storageGroupMNode to databaseMNode; adds checkTableDevice4Object traversal method
SchemaRegionPBTreeImpl.java Adds stub checkTableDevice4Object method throwing UnsupportedOperationException
SchemaRegionMemoryImpl.java Implements checkTableDevice4Object by delegating to MTree
ISchemaRegion.java Adds checkTableDevice4Object interface method
TableHeaderSchemaValidator.java Updates lock type to AVOID_CONCURRENT_DEVICE_ALTER_TABLE
TableDeviceSchemaValidator.java Changes exception type from IoTDBException to IoTDBRuntimeException
StatementAnalyzer.java Validates device IDs against OBJECT field restrictions during CREATE OR UPDATE DEVICE
ClusterConfigTaskExecutor.java Handles OBJECT field validation errors during ALTER TABLE ADD COLUMN
TableConfigTaskVisitor.java Validates table and object names when creating tables with OBJECT columns
SchemaLockType.java Renames VALIDATE_VS_DELETION_TABLE to AVOID_CONCURRENT_DEVICE_ALTER_TABLE
DataNodeInternalRPCServiceImpl.java Implements checkDeviceIdForObject RPC handler with lock protection
AddTableColumnProcedure.java Adds checkObject method and parseStatus helper for validation when adding columns
AbstractAlterOrDropTableProcedure.java Enhances TableRegionTaskExecutor with configurable exception generator
ClusterSchemaManager.java Validates table and object names during column extension
DataNodeAsyncRequestRPCHandler.java Registers CHECK_DEVICE_ID_FOR_OBJECT handler
CnToDnInternalServiceAsyncRequestManager.java Maps CHECK_DEVICE_ID_FOR_OBJECT to checkDeviceIdForObject client call
CnToDnAsyncRequestType.java Adds CHECK_DEVICE_ID_FOR_OBJECT enum value
IoTDBTableIT.java Adds comprehensive integration test for OBJECT field validation
Comments suppressed due to low confidence (1)

iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/execution/config/executor/ClusterConfigTaskExecutor.java:4224

  • The error message parsing using string contains and replace is fragile. If the exception message format changes, this code will break silently. Consider using a specific error code in TSStatus or a more robust parsing mechanism to identify and handle OBJECT field validation errors.
        ReadWriteIOUtils.write(targetName, stream);
      } catch (final IOException ignored) {
        // ByteArrayOutputStream won't throw IOException
      }

      final TSStatus tsStatus =

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +168 to +171
String message = "";
for (final TSStatus status : statuses) {
if (status.getCode() == TSStatusCode.SEMANTIC_ERROR.getStatusCode()) {
message = status.getMessage();
Copy link

Copilot AI Nov 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Initializing 'message' to empty string and then reassigning it in the loop is inefficient. Consider initializing to null and using Objects.isNull() checks, or restructure to return early when a semantic error is found, to avoid unnecessary string assignments.

Copilot uses AI. Check for mistakes.
Comment on lines +144 to +166
private void checkObject(
final ConfigNodeProcedureEnv env, final String database, final String tableName) {
final Map<TConsensusGroupId, TRegionReplicaSet> relatedRegionGroup =
env.getConfigManager().getRelatedSchemaRegionGroup4TableModel(database);

if (!relatedRegionGroup.isEmpty()) {
new TableRegionTaskExecutor<>(
"check deviceId for object",
env,
relatedRegionGroup,
CnToDnAsyncRequestType.CHECK_DEVICE_ID_FOR_OBJECT,
((dataNodeLocation, consensusGroupIdList) ->
new TCheckDeviceIdForObjectReq(new ArrayList<>(consensusGroupIdList), tableName)),
((tConsensusGroupId, tDataNodeLocations, failureMap) -> {
final String message = parseStatus(failureMap.values());
// Shall not be SUCCESS here
return Objects.nonNull(message)
? new IoTDBException(message, TSStatusCode.SEMANTIC_ERROR.getStatusCode())
: null;
}))
.execute();
}
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why check while adding a column?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When adding a object column, we need to ensure that the directory parts does not contain illegal paths

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can do this during schema validation of insertion. No need to do so in adding columns.
If an existing device will not be inserted with object data, you do not have to fail the column addition.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants