Skip to content

fix: Set DatabricksConnectionConfig to only use a shared_connection with U2M OAuth#5859

Open
davem-bis wants to merge 2 commits into
SQLMesh:mainfrom
davem-bis:feature/DRM/revert-to-shared-connection-for-databricks-access-token-connections
Open

fix: Set DatabricksConnectionConfig to only use a shared_connection with U2M OAuth#5859
davem-bis wants to merge 2 commits into
SQLMesh:mainfrom
davem-bis:feature/DRM/revert-to-shared-connection-for-databricks-access-token-connections

Conversation

@davem-bis

@davem-bis davem-bis commented Jun 24, 2026

Copy link
Copy Markdown
Contributor

Description

The DatabricksConnectionConfig was set to use a shared connection pool for all authentication types to resolve a race condition with U2M OAuth, however this has the side effect that SQL execution is sequential regardless of the concurrent_tasks config setting.

This change updates:

  • The shared_connection variable to be a computed field for all ConnectionConfig classes.
  • The shared_connection variable to return True only for U2M OAuth authentication methods for the DatabricksConnectionConfig class.

Partially resolves #5858.

Test Plan

  • Unit testing for all 3 situations
  • Tested on Python 3.12.3, SQLMesh 0.253.3, databricks-sql-connector 4.0.5 with Access Token and U2M OAuth.

Checklist

  • I have run make style and fixed any issues
  • I have added tests for my changes (if applicable)
  • All existing tests pass (make fast-test)
  • My commits are signed off (git commit -s) per the DCO

…hen U2M OAuth authentication is used.

Signed-off-by: davem-bis <68955845+davem-bis@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bug(Databricks): databricks-sql-connector executes queries sequentially regardless of concurrent_tasks

1 participant