diff --git a/README.md b/README.md index 645540017b21..4d797bdbdf0e 100644 --- a/README.md +++ b/README.md @@ -4,6 +4,10 @@ * [Java on Google Cloud][cloud-java] +## Documentation + +See the [official guide](https://cloud.google.com/java/docs/setup) to get setup and started with development. + ## Supported APIs Libraries are available on GitHub and Maven Central for developing Java applications that interact with individual Google Cloud services: @@ -236,371 +240,6 @@ If the service is not listed, [google-api-java-client][google-api-java-client-se *When building Java applications, preference should be given to the libraries listed in the table.* - - -## Specifying a Project ID - -Most `google-cloud` libraries require a project ID. There are multiple ways to specify this project ID. - -1. When using `google-cloud` libraries from within Compute/App Engine, there's no need to specify a project ID. It is automatically inferred from the production environment. -2. When using `google-cloud` elsewhere, you can do one of the following: -* Supply the project ID when building the service options. For example, to use Datastore from a project with ID "PROJECT_ID", you can write: - - ```java - Datastore datastore = DatastoreOptions.newBuilder().setProjectId("PROJECT_ID").build().getService(); - ``` -* Specify the environment variable `GOOGLE_CLOUD_PROJECT` to be your desired project ID. -* Set the project ID using the [Google Cloud SDK](https://cloud.google.com/sdk/?hl=en). To use the SDK, [download the SDK](https://cloud.google.com/sdk/?hl=en) if you haven't already, and set the project ID from the command line. For example: - - ``` - gcloud config set project PROJECT_ID - ``` - -`google-cloud` determines the project ID from the following sources in the listed order, stopping once it finds a value: - -1. The project ID supplied when building the service options -2. Project ID specified by the environment variable `GOOGLE_CLOUD_PROJECT` -3. The App Engine / Compute Engine project ID -4. The project ID specified in the JSON credentials file pointed by the `GOOGLE_APPLICATION_CREDENTIALS` environment variable -5. The Google Cloud SDK project ID - -In cases where the library may expect a project ID explicitly, we provide a helper that can provide the inferred project ID: - ```java - import com.google.cloud.ServiceOptions; - ... - String projectId = ServiceOptions.getDefaultProjectId(); - ``` - -## Authentication - -`google-cloud-java` uses -[https://github.com/googleapis/google-auth-library-java](https://github.com/googleapis/google-auth-library-java) -to authenticate requests. `google-auth-library-java` supports a wide range of authentication types; -see the project's [README](https://github.com/google/google-auth-library-java/blob/main/README.md) -and [javadoc](https://cloud.google.com/java/docs/reference/google-auth-library/latest/overview) for more -details. - -### Google Cloud Platform environment - -When using Google Cloud libraries from a Google Cloud Platform environment such as Compute Engine, -Kubernetes Engine, or App Engine, no additional authentication steps are necessary. - -For example: - -```java -Storage storage = StorageOptions.getDefaultInstance().getService(); -``` - -or: - -```java -CloudTasksClient cloudTasksClient = CloudTasksClient.create(); -``` - -### Other environments - -#### Using a service account (recommended) - -1. [Generate a JSON service account key](https://cloud.google.com/storage/docs/authentication?hl=en#service_accounts). - -2. After downloading that key, you must do one of the following: - * Define the environment variable GOOGLE_APPLICATION_CREDENTIALS to be the location of the key. - For example: - ```bash - export GOOGLE_APPLICATION_CREDENTIALS=/path/to/my/key.json - ``` - * Supply the JSON credentials file when building the service options. For example, this Storage - object has the necessary permissions to interact with your Google Cloud Storage data: - ```java - Storage storage = StorageOptions.newBuilder() - .setCredentials(ServiceAccountCredentials.fromStream(new FileInputStream("/path/to/my/key.json"))) - .build() - .getService(); - ``` - -#### Local development/testing - -If running locally for development/testing, you can use the [Google Cloud SDK](https://cloud.google.com/sdk/). -Create Application Default Credentials with `gcloud auth application-default login`, and then -`google-cloud` will automatically detect such credentials. - -#### Existing OAuth2 access token - -If you already have an OAuth2 access token, you can use it to authenticate (notice that in this case, the -access token will not be automatically refreshed): - -```java -Credentials credentials = GoogleCredentials.create(new AccessToken(accessToken, expirationTime)); -Storage storage = StorageOptions.newBuilder() - .setCredentials(credentials) - .build() - .getService(); -``` - -or: - -```java -Credentials credentials = GoogleCredentials.create(new AccessToken(accessToken, expirationTime)); -CloudTasksSettings cloudTasksSettings = CloudTasksSettings.newBuilder() - .setCredentialProvider(FixedCredentialsProvider.create(credentials)) - .build(); -CloudTasksClient cloudTasksClient = CloudTasksClient.create(cloudTasksSettings); -``` - -### Application Default Credentials - -If no credentials are provided, `google-cloud` will attempt to detect them from the environment -using `GoogleCredentials.getApplicationDefault()` which will search for Application Default -Credentials in the following locations (in order): - -1. The credentials file pointed to by the `GOOGLE_APPLICATION_CREDENTIALS` environment variable -2. Credentials provided by the Google Cloud SDK `gcloud auth application-default login` command -3. Google App Engine built-in credentials -4. Google Cloud Shell built-in credentials -5. Google Compute Engine built-in credentials - -### Authenticating with an API Key - -[Authenticating with API Keys](https://cloud.google.com/docs/authentication/api-keys) is supported by a handful of Google Cloud APIs. - -We are actively exploring ways to improve the API Key experience. -Currently, to use an API Key with a Java client library, you need to set the header for the relevant service Client manually. - -For example, to set the API Key with the [Language service](https://cloud.google.com/java/docs/reference/google-cloud-language/latest/overview): - -```java -public LanguageServiceClient createGrpcClientWithApiKey(String apiKey) throws Exception { - // Manually set the api key via the header - Map header = new HashMap() { {put("x-goog-api-key", apiKey);}}; - FixedHeaderProvider headerProvider = FixedHeaderProvider.create(header); - - // Create the client - TransportChannelProvider transportChannelProvider = InstantiatingGrpcChannelProvider.newBuilder().setHeaderProvider(headerProvider).build(); - LanguageServiceSettings settings = LanguageServiceSettings.newBuilder().setTransportChannelProvider(transportChannelProvider).build(); - LanguageServiceClient client = LanguageServiceClient.create(settings); - return client; - } -``` - -An example instantiation with the Language Client using rest: -```java - public LanguageServiceClient createRestClientWithApiKey(String apiKey) throws Exception { - // Manually set the api key header - Map header = new HashMap() { {put("x-goog-api-key", apiKey);}}; - FixedHeaderProvider headerProvider = FixedHeaderProvider.create(header); - - // Create the client - TransportChannelProvider transportChannelProvider = InstantiatingHttpJsonChannelProvider.newBuilder().setHeaderProvider(headerProvider).build(); - LanguageServiceSettings settings = LanguageServiceSettings.newBuilder().setTransportChannelProvider(transportChannelProvider).build(); - LanguageServiceClient client = LanguageServiceClient.create(settings); - return client; - } -``` - -## Troubleshooting - -To get help, follow the instructions in the [Troubleshooting document](https://github.com/googleapis/google-cloud-java/blob/main/TROUBLESHOOTING.md). - -## Configuring a Proxy - -Google Cloud client libraries use HTTPS and gRPC in underlying communication -with the services. -In both protocols, you can configure a proxy using `https.proxyHost` -and (optional) `https.proxyPort` properties. - -### gRPC Custom Proxy Configuration - -For a more custom proxy with gRPC, you will need supply a `ProxyDetector` to -the `ManagedChannelBuilder`: - -```java -import com.google.api.core.ApiFunction; -import com.google.api.gax.rpc.TransportChannelProvider; -import com.google.cloud.tasks.v2.CloudTasksClient; -import com.google.cloud.tasks.v2.CloudTasksSettings; -import com.google.cloud.tasks.v2.stub.CloudTasksStubSettings; -import io.grpc.HttpConnectProxiedSocketAddress; -import io.grpc.ManagedChannelBuilder; -import io.grpc.ProxiedSocketAddress; -import io.grpc.ProxyDetector; - -import javax.annotation.Nullable; -import java.io.IOException; -import java.net.InetSocketAddress; -import java.net.SocketAddress; - -public CloudTasksClient getService() throws IOException { - TransportChannelProvider transportChannelProvider = - CloudTasksStubSettings.defaultGrpcTransportProviderBuilder() - .setChannelConfigurator( - new ApiFunction() { - @Override - public ManagedChannelBuilder apply(ManagedChannelBuilder managedChannelBuilder) { - return managedChannelBuilder.proxyDetector( - new ProxyDetector() { - @Nullable - @Override - public ProxiedSocketAddress proxyFor(SocketAddress socketAddress) - throws IOException { - return HttpConnectProxiedSocketAddress.newBuilder() - .setUsername(PROXY_USERNAME) - .setPassword(PROXY_PASSWORD) - .setProxyAddress(new InetSocketAddress(PROXY_HOST, PROXY_PORT)) - .setTargetAddress((InetSocketAddress) socketAddress) - .build(); - } - }); - } - }) - .build(); - CloudTasksSettings cloudTasksSettings = - CloudTasksSettings.newBuilder() - .setTransportChannelProvider(transportChannelProvider) - .build(); - return CloudTasksClient.create(cloudTasksSettings); -} -``` - -## Long Running Operations - -Long running operations (LROs) are often used for API calls that are expected to -take a long time to complete (i.e. provisioning a GCE instance or a Dataflow pipeline). -The initial API call creates an "operation" on the server and returns an Operation ID -to track its progress. LRO RPCs have the suffix `Async` appended to the call name -(i.e. `clusterControllerClient.createClusterAsync()`) - -Our generated clients provide a nice interface for starting the operation and -then waiting for the operation to complete. This is accomplished by returning an -[`OperationFuture`](https://cloud.google.com/java/docs/reference/gax/latest/com.google.api.gax.longrunning.OperationFuture). -When calling `get()` on the `OperationFuture`, the client library will poll the operation to -check the operation's status. - -For example, take a sample `createCluster` Operation in google-cloud-dataproc v4.20.0: -```java -try (ClusterControllerClient clusterControllerClient = ClusterControllerClient.create()) { - CreateClusterRequest request = - CreateClusterRequest.newBuilder() - .setProjectId("{PROJECT_ID}") - .setRegion("{REGION}") - .setCluster(Cluster.newBuilder().build()) - .setRequestId("{REQUEST_ID}") - .setActionOnFailedPrimaryWorkers(FailureAction.forNumber(0)) - .build(); - OperationFuture future = - clusterControllerClient.createClusterOperationCallable().futureCall(request); - // Do something. - Cluster response = future.get(); -} catch (CancellationException e) { - // Exceeded the default RPC timeout without the Operation completing. - // Library is no longer polling for the Operation status. Consider - // increasing the timeout. -} -``` - -### LRO Timeouts -The polling operations have a default timeout that varies from service to service. -The library will throw a `java.util.concurrent.CancellationException` with the message: -`Task was cancelled.` if the timeout exceeds the operation. A `CancellationException` -does not mean that the backend GCP Operation was cancelled. This exception is thrown from the -client library when it has exceeded the total timeout without receiving a successful status from the operation. -Our client libraries respect the configured values set in the OperationTimedPollAlgorithm for each RPC. - -Note: The client library handles the Operation's polling mechanism for you. By default, there is no need -to manually poll the status yourself. - -### Default LRO Values -Each LRO RPC has a set of pre-configured default values. You can find these values by -searching in each Client's `StubSettings`'s class. The default LRO settings are initialized -inside the `initDefaults()` method in the nested Builder class. - -For example, in google-cloud-aiplatform v3.24.0, the default [OperationTimedPollAlgorithm](https://github.com/googleapis/google-cloud-java/blob/9ae786d1acdc7354adf86b78691570668caa293d/java-aiplatform/google-cloud-aiplatform/src/main/java/com/google/cloud/aiplatform/v1/stub/EndpointServiceStubSettings.java#L755-L765) -has these default values: -```java -OperationTimedPollAlgorithm.create( - RetrySettings.newBuilder() - .setInitialRetryDelay(Duration.ofMillis(5000L)) - .setRetryDelayMultiplier(1.5) - .setMaxRetryDelay(Duration.ofMillis(45000L)) - .setInitialRpcTimeout(Duration.ZERO) - .setRpcTimeoutMultiplier(1.0) - .setMaxRpcTimeout(Duration.ZERO) - .setTotalTimeout(Duration.ofMillis(300000L)) - .build()) -``` -Both retries and LROs share the same RetrySettings class. Note the corresponding link: -- Total Timeout (Max Time allowed for polling): 5 minutes -- Initial Retry Delay (Initial delay before first poll): 5 seconds -- Max Retry Delay (Maximum delay between each poll): 45 seconds -- Retry Delay Multiplier (Multiplier value to increase the poll delay): 1.5 - -The RPC Timeout values have no use in LROs and can be omitted or set to the default values -(`Duration.ZERO` for Timeouts or `1.0` for the multiplier). - -### Configuring LRO Timeouts -To configure the LRO values, create an OperationTimedPollAlgorithm object and update the -RPC's polling algorithm. For example: -```java -ClusterControllerSettings.Builder settingsBuilder = ClusterControllerSettings.newBuilder(); -TimedRetryAlgorithm timedRetryAlgorithm = OperationTimedPollAlgorithm.create( - RetrySettings.newBuilder() - .setInitialRetryDelay(Duration.ofMillis(500L)) - .setRetryDelayMultiplier(1.5) - .setMaxRetryDelay(Duration.ofMillis(5000L)) - .setInitialRpcTimeout(Duration.ZERO) // ignored - .setRpcTimeoutMultiplier(1.0) // ignored - .setMaxRpcTimeout(Duration.ZERO) // ignored - .setTotalTimeout(Duration.ofHours(24L)) // set polling timeout to 24 hours - .build()); -settingsBuilder.createClusterOperationSettings() - .setPollingAlgorithm(timedRetryAlgorithm); -ClusterControllerClient clusterControllerClient = ClusterControllerClient.create(settingsBuilder.build()); -``` - -Note: The configuration above *only* modifies the LRO values for the `createClusterOperation` RPC. -The other RPCs in the Client will still use each RPC's pre-configured LRO values. - -## Managing Dependencies - -If you are using more than one Google Cloud client library, we recommend you use one of -our Bill of Material (BOM) artifacts to help manage dependency versions. For more information, -see [Using the Cloud Client Libraries](https://cloud.google.com/java/docs/bom). - -## Java Versions - -Java 8 or above is required for using the clients in this repository. - -## Supported Platforms - -Clients in this repository use either HTTP or gRPC for the transport layer. All -HTTP-based clients should work in all environments. - -For clients that use gRPC, the supported platforms are constrained by the platforms -that [Forked Tomcat Native](http://netty.io/wiki/forked-tomcat-native.html) supports, -which for architectures means only x86_64, and for operating systems means Mac OS X, -Windows, and Linux. Additionally, gRPC constrains the use of platforms with -threading restrictions. - -Thus, the following are not supported: - -- Android - - Consider [Firebase](https://firebase.google.com), which includes many of these APIs. - - It is possible to use these libraries in many cases, although it is unsupported. - You can find examples, such as [this one](https://github.com/GoogleCloudPlatform/android-docs-samples/tree/master/speech/SpeechRecognitionClient), - in this [example repository](https://github.com/GoogleCloudPlatform/android-docs-samples) but consider the risks carefully before using these libraries in an application. -- Raspberry Pi (since it runs on the ARM architecture) -- Google App Engine Standard Java 7 - -The following environments should work (among others): - -- standalone Windows on x86_64 -- standalone Mac OS X on x86_64 -- standalone Linux on x86_64 -- Google Compute Engine (GCE) -- Google Container Engine (GKE) -- Google App Engine Standard Java 8 (GAE Std J8) -- Google App Engine Flex (GAE Flex) -- Alpine Linux (Java 11+) - ## Testing This library provides tools to help write tests for code that uses google-cloud services.