Update README #805
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,148 +1,126 @@ | ||
| # Databricks JDBC Driver | ||
|
|
||
| The Databricks JDBC driver implements the JDBC interface providing connectivity to a Databricks SQL warehouse. | ||
| Please refer to [Databricks documentation](https://docs.databricks.com/aws/en/integrations/jdbc-oss/) for more | ||
| information. | ||
|
|
||
| [](https://opensource.org/licenses/Apache-2.0) | ||
|
|
||
| ## Prerequisites | ||
|
|
||
| Databricks JDBC is compatible with Java 11 and higher. CI testing runs on Java versions 11, 17, and 21. | ||
|
|
||
| ## Installation | ||
|
|
||
| ### Maven | ||
|
|
||
| Add the following dependency to your `pom.xml`: | ||
|
|
||
| ```xml | ||
| <dependency> | ||
| <groupId>com.databricks</groupId> | ||
| <artifactId>databricks-jdbc</artifactId> | ||
| <version>1.0.4-oss</version> | ||
| </dependency> | ||
| ``` | ||
|
Collaborator
There was a problem hiding this comment. is this covered anywhere? else we can still keep the running part and move full documentation to a confluence page
Collaborator
Author
There was a problem hiding this comment. There is a separate page for integration test in docs folder |
||
|
|
||
| ### Build from Source | ||
|
|
||
| 1. Clone the repository | ||
| 2. Run the following command: | ||
| ```bash | ||
| mvn clean package | ||
| ``` | ||
| 3. The jar file is generated as `target/databricks-jdbc-<version>.jar` | ||
| 4. The test coverage report is generated in `target/site/jacoco/index.html` | ||
|
|
||
| ## Usage | ||
|
|
||
| ### Connection String | ||
|
|
||
| ``` | ||
| jdbc:databricks://<host>:<port>;transportMode=http;ssl=1;AuthMech=3;httpPath=<path>;UID=token;PWD=<token> | ||
| ``` | ||
|
|
||
| ### Authentication | ||
|
Collaborator
There was a problem hiding this comment. We can add public documentation link for config properties if one wants to go into more details on Auth
Collaborator
Author
There was a problem hiding this comment. It seems that our public documentation doesn't include all authentication types, and we also plan to exclude them from the README. Therefore, I believe this comment is now unnecessary. |
||
|
|
||
| The JDBC driver supports the following authentication methods: | ||
|
|
||
| #### Personal Access Token (PAT) | ||
|
|
||
| Use `AuthMech=3` for personal access token authentication: | ||
|
|
||
| ``` | ||
| AuthMech=3;UID=token;PWD=<your_token> | ||
| ``` | ||
|
|
||
| #### OAuth2 Authentication | ||
|
|
||
| Use `AuthMech=11` for OAuth2-based authentication. Several OAuth flows are supported: | ||
|
|
||
| ##### Token Passthrough | ||
|
|
||
| Direct use of an existing OAuth token: | ||
|
|
||
| ``` | ||
| AuthMech=11;Auth_Flow=0;Auth_AccessToken=<your_access_token> | ||
| ``` | ||
|
|
||
| ##### OAuth Client Credentials (Machine-to-Machine) | ||
|
|
||
| Configure standard OAuth client credentials flow: | ||
|
|
||
| ``` | ||
| AuthMech=11;Auth_Flow=1;OAuth2ClientId=<client_id>;OAuth2Secret=<client_secret> | ||
| ``` | ||
|
|
||
| Optional parameters: | ||
| - `AzureTenantId`: Azure tenant ID for Azure Databricks (default: null). If enabled, the driver will include refreshed | ||
| Azure Active Directory (AAD) Service Principal OAuth tokens with every request. | ||
|
|
||
| ##### Browser-Based OAuth | ||
|
|
||
| Interactive browser-based OAuth flow with PKCE: | ||
|
|
||
| ``` | ||
| AuthMech=11;Auth_Flow=2 | ||
| ``` | ||
|
|
||
| Optional parameters: | ||
| - `OAuth2ClientId` - Client ID for OAuth2 (default: databricks-cli) | ||
| - `OAuth2RedirectUrlPort` - Ports for redirect URL (default: 8020) | ||
| - `EnableOIDCDiscovery` - Enable OIDC discovery (default: 1) | ||
| - `OAuthDiscoveryURL` - OIDC discovery endpoint (default: /oidc/.well-known/oauth-authorization-server) | ||
|
|
||
| ### Logging | ||
|
|
||
| The driver supports both SLF4J and Java Util Logging (JUL) frameworks: | ||
|
|
||
| - **SLF4J**: Enable with `-Dcom.databricks.jdbc.loggerImpl=SLF4JLOGGER` | ||
| - **JUL**: Enable with `-Dcom.databricks.jdbc.loggerImpl=JDKLOGGER` (default) | ||
|
|
||
| For detailed logging configuration options, see [Logging Documentation](./docs/logging.md). | ||
|
|
||
| ## Running Tests | ||
|
Collaborator
There was a problem hiding this comment. We can also add JVM property for nio under using the driver section
Collaborator
There was a problem hiding this comment. will this cover running fake service tests also?
Collaborator
Author
There was a problem hiding this comment. For fake service tests, I have added a separate detailed page in docs folder |
||
|
|
||
| Basic test execution: | ||
|
|
||
| ```bash | ||
| mvn test | ||
| ``` | ||
|
|
||
| **Note**: Due to a change in JDK 16 that introduced a compatibility issue with the Apache Arrow library used by the JDBC | ||
| driver, runtime errors may occur when using the JDBC driver with JDK 16 or later. To avoid these errors, restart your | ||
| application or driver with the following JVM command option: | ||
|
|
||
| ``` | ||
| --add-opens=java.base/java.nio=org.apache.arrow.memory.core ALL-UNNAMED | ||
| ``` | ||
|
|
||
| For more detailed information about integration tests and fake services, see [Testing Documentation](./docs/testing.md). | ||
|
|
||
| ## Documentation | ||
|
|
||
| For more information, see the following resources: | ||
| - [Integration Tests Guide](./docs/testing.md) | ||
| - [Logging Configuration](./docs/logging.md) | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,94 @@ | ||
| # Logging Configuration | ||
|
|
||
| The Databricks JDBC driver supports both [SLF4J](https://www.slf4j.org/) and [JUL](https://docs.oracle.com/javase/8/docs/api/java/util/logging/package-summary.html) logging frameworks. | ||
|
|
||
| ## SLF4J Logging | ||
|
|
||
| SLF4J logging can be enabled by setting the system property: | ||
| ``` | ||
| -Dcom.databricks.jdbc.loggerImpl=SLF4JLOGGER | ||
| ``` | ||
|
|
||
| You need to provide an SLF4J binding implementation and corresponding configuration file in the classpath. This gives you the freedom to adapt the JDBC logging to your specific needs. | ||
|
|
||
| ### Example: Using SLF4J with Log4j2 | ||
|
|
||
| Add the following dependencies to your `pom.xml`: | ||
|
|
||
| ```xml | ||
| <dependency> | ||
| <groupId>org.apache.logging.log4j</groupId> | ||
| <artifactId>log4j-slf4j2-impl</artifactId> | ||
| <version>${log4j.version}</version> | ||
| </dependency> | ||
| <dependency> | ||
| <groupId>org.apache.logging.log4j</groupId> | ||
| <artifactId>log4j-core</artifactId> | ||
| <version>${log4j.version}</version> | ||
| </dependency> | ||
| <dependency> | ||
| <groupId>org.apache.logging.log4j</groupId> | ||
| <artifactId>log4j-api</artifactId> | ||
| <version>${log4j.version}</version> | ||
| </dependency> | ||
| ``` | ||
|
|
||
| Create a `log4j2.xml` configuration file: | ||
|
|
||
| ```xml | ||
| <?xml version="1.0" encoding="UTF-8"?> | ||
| <Configuration status="WARN"> | ||
| <Appenders> | ||
| <!-- Console appender for default logging --> | ||
| <Console name="Console" target="SYSTEM_OUT"> | ||
| <PatternLayout pattern="%d{yyyy-MM-dd HH:mm:ss} %-5level %logger{36} - %msg%n"/> | ||
| </Console> | ||
| </Appenders> | ||
|
|
||
| <Loggers> | ||
| <!-- Root logger to catch any logs that don't match other loggers --> | ||
| <Root level="info"> | ||
| <AppenderRef ref="Console"/> | ||
| </Root> | ||
| </Loggers> | ||
| </Configuration> | ||
| ``` | ||
|
|
||
| ## Java Util Logging (JUL) | ||
|
|
||
| JUL logging is enabled by default, or can be explicitly set with: | ||
| ``` | ||
| -Dcom.databricks.jdbc.loggerImpl=JDKLOGGER | ||
| ``` | ||
|
|
||
| There are two ways to configure JUL logging: | ||
|
|
||
| ### 1. JDBC URL Parameters | ||
|
|
||
| Standard logging parameters can be passed in the JDBC URL: | ||
|
|
||
| ``` | ||
| jdbc:databricks://your-databricks-host:443;transportMode=http;ssl=1;AuthMech=3;httpPath=/sql/1.0/warehouses/your-warehouse-id;UID=token;logLevel=DEBUG;logPath=/path/to/dir;logFileSize=10;logFileCount=5 | ||
| ``` | ||
|
|
||
| Available parameters: | ||
| - `logLevel`: Logging level (e.g., DEBUG, INFO) | ||
| - `logPath`: Directory path for log files | ||
| - `logFileSize`: Maximum size of each log file in MB | ||
| - `logFileCount`: Maximum number of log files to keep | ||
|
|
||
| ### 2. Configuration File | ||
|
|
||
| Logging properties can also be set in a `logging.properties` file in the classpath: | ||
|
|
||
| ```properties | ||
| handlers=java.util.logging.FileHandler, java.util.logging.ConsoleHandler | ||
| .level=INFO | ||
| java.util.logging.FileHandler.level=ALL | ||
| java.util.logging.FileHandler.pattern=/path/to/dir/databricks-jdbc.log | ||
| java.util.logging.FileHandler.limit=10000000 | ||
| java.util.logging.FileHandler.count=5 | ||
| java.util.logging.FileHandler.formatter=java.util.logging.SimpleFormatter | ||
| java.util.logging.ConsoleHandler.level=ALL | ||
| java.util.logging.ConsoleHandler.formatter=java.util.logging.SimpleFormatter | ||
| ``` |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think we should add our public documentation link here for more details.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done