Skip to content

Conversation

@jadewang-db
Copy link
Contributor

@jadewang-db jadewang-db commented Oct 17, 2025

Summary

This PR implements the foundational configuration and data models for integrating the Databricks Statement
Execution REST API into the ADBC driver. This is the first subtask in a multi-phase effort to support the new
REST-based protocol as an alternative to the existing Thrift/HiveServer2 implementation.

Changes

Configuration Parameters (DatabricksParameters.cs):

  • Added 6 new ADBC parameters for Statement Execution API control:
    • Protocol: Select between thrift (default) or rest protocols
    • ResultDisposition: Control result delivery (inline, external_links, or inline_or_external_links)
    • ResultFormat: Choose result format (arrow_stream, json_array, or csv)
    • ResultCompression: Set compression codec (lz4, gzip, or none)
    • WaitTimeout: Configure statement execution wait timeout in seconds (default: 10s)
    • PollingInterval: Set polling interval for async execution in milliseconds (default: 1000ms)
  • Added 4 constants classes for protocol types, result dispositions, formats, and compressions
  • All parameters follow adbc.databricks.* naming convention

REST API Models (StatementExecution/StatementExecutionModels.cs):

  • Created 17 model classes with System.Text.Json serialization attributes:
    • Session management: CreateSessionRequest, CreateSessionResponse
    • Statement execution: ExecuteStatementRequest, ExecuteStatementResponse, GetStatementResponse
    • Status and errors: StatementStatus, StatementError
    • Result data: ResultManifest, ResultChunk, ExternalLink, ResultSchema, ColumnInfo
    • Parameters and data: StatementParameter, StatementResult, ResultData
  • Organized in new StatementExecution namespace for clean separation
  • Supports both inline and external link result dispositions

Test Coverage:

  • StatementExecutionParametersTest.cs: 8 test methods validating parameter constants, naming conventions, and
    default values
  • StatementExecutionModelsTest.cs: 15 test methods for JSON serialization/deserialization of all model types
  • All 23 new tests passing ✅

Jade Wang added 9 commits October 14, 2025 11:40
Implements PECO-2787: Configuration & Models for Statement Execution API support.

Added 6 new configuration parameters:
- Protocol: choose between thrift and rest protocols
- ResultDisposition: inline, external_links, or hybrid mode
- ResultFormat: arrow_stream, json_array, or csv
- ResultCompression: lz4, gzip, or none
- WaitTimeout: statement execution wait timeout in seconds
- PollingInterval: polling interval for async execution in milliseconds

Created 17 REST API model classes with JSON serialization in new
StatementExecution namespace for session management, statement execution,
result manifests, and external link handling.

Added comprehensive test coverage with 23 unit tests across 2 test files
validating parameter constants and model serialization/deserialization.

All tests passing (31 total including related tests).
@github-actions github-actions bot added this to the ADBC Libraries 21 milestone Oct 17, 2025
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this file is duplicate with another PR, will merge and resolve after the other PR merged.

@jadewang-db jadewang-db changed the title fix(csharp/src/Drivers/Databricks): add configuration parameters and models for SEA feat(csharp/src/Drivers/Databricks): add configuration parameters and models for SEA Oct 17, 2025
@jadewang-db
Copy link
Contributor Author

there is a sessions/ path, that you can create session api.

@eric-wang-1990
Copy link
Contributor

there is a sessions/ path, that you can create session api.

Is that a public path? I am not seeing it in the public doc.
Are we going to create a session first? It's better if we can do that, so all the execute statement can share the same session.

@jadewang-db
Copy link
Contributor Author

there is a sessions/ path, that you can create session api.

Is that a public path? I am not seeing it in the public doc. Are we going to create a session first? It's better if we can do that, so all the execute statement can share the same session.

https://github.com/databricks/databricks-jdbc/blob/main/src/main/java/com/databricks/jdbc/model/client/sqlexec/CreateSessionRequest.java

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants