I have Data in BigQuery and trying to migrate that to BigTable using BigQueryBigtableTransfer DF template.
running it like
mvn package exec:exec \
-DBigQueryBigtableTransfer \
-Dbigtable.projectID=google-cloud-project-id \
-Dbigtable.instanceID=Big-table-instance-id-dev \
-Dgs=gs://google-cloud-project-id-bigtable/bigtable_table/ \
-Dbq.query='SELECT * FROM `google-cloud-project-id.SAMPLE.bigtable_test`' \
-Dregion=us-west1
With or without region still gives error of region parameter missing even if region paramater is no where mentioned in redme file.
getting below error -
[INFO] --- exec-maven-plugin:1.4.0:exec (default-cli) @ dataflow-connector-examples ---
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/C:/Users/xyz/.m2/repository/org/slf4j/slf4j-log4j12/1.7.25/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/C:/Users/xyz/.m2/repository/org/slf4j/slf4j-simple/1.7.21/slf4j-simple-1.7.21.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
log4j:WARN No appenders could be found for logger (org.apache.beam.sdk.options.PipelineOptionsFactory).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
Exception in thread "main" java.lang.RuntimeException: Failed to construct instance from factory method DataflowRunner#fromOptions(interface org.apache.beam.sdk.options.PipelineOptions)
at org.apache.beam.sdk.util.InstanceBuilder.buildFromMethod(InstanceBuilder.java:224)
at org.apache.beam.sdk.util.InstanceBuilder.build(InstanceBuilder.java:155)
at org.apache.beam.sdk.PipelineRunner.fromOptions(PipelineRunner.java:55)
at org.apache.beam.sdk.Pipeline.create(Pipeline.java:149)
at com.google.cloud.bigtable.dataflow.example.BigQueryBigtableTransfer.main(BigQueryBigtableTransfer.java:118)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.beam.sdk.util.InstanceBuilder.buildFromMethod(InstanceBuilder.java:214)
... 4 more
Caused by: java.lang.IllegalArgumentException: Missing required pipeline options: region
at org.apache.beam.runners.dataflow.DataflowRunner.fromOptions(DataflowRunner.java:262)
... 9 more
Can you please advice whats wrong?
I have Data in BigQuery and trying to migrate that to BigTable using
BigQueryBigtableTransferDF template.running it like
With or without region still gives error of region parameter missing even if region paramater is no where mentioned in redme file.
getting below error -
Can you please advice whats wrong?