Skip to content

Commit faa0d07

Browse files
authored
Bump Flint version to 0.3.0 (#258)
Signed-off-by: Peng Huo <[email protected]>
1 parent be8cb32 commit faa0d07

File tree

6 files changed

+13
-9
lines changed

6 files changed

+13
-9
lines changed

.gitignore

+2
Original file line numberDiff line numberDiff line change
@@ -19,3 +19,5 @@ logs/
1919
*.class
2020
*.log
2121
*.zip
22+
23+
gen/

README.md

+3-2
Original file line numberDiff line numberDiff line change
@@ -19,6 +19,7 @@ Version compatibility:
1919
|---------------|-------------|---------------|---------------|------------|
2020
| 0.1.0 | 11+ | 3.3.1 | 2.12.14 | 2.6+ |
2121
| 0.2.0 | 11+ | 3.3.1 | 2.12.14 | 2.6+ |
22+
| 0.3.0 | 11+ | 3.3.2 | 2.12.14 | 2.6+ |
2223

2324
## Flint Extension Usage
2425

@@ -50,7 +51,7 @@ sbt clean standaloneCosmetic/publishM2
5051
```
5152
then add org.opensearch:opensearch-spark_2.12 when run spark application, for example,
5253
```
53-
bin/spark-shell --packages "org.opensearch:opensearch-spark_2.12:0.2.0-SNAPSHOT"
54+
bin/spark-shell --packages "org.opensearch:opensearch-spark_2.12:0.3.0-SNAPSHOT"
5455
```
5556

5657
### PPL Build & Run
@@ -62,7 +63,7 @@ sbt clean sparkPPLCosmetic/publishM2
6263
```
6364
then add org.opensearch:opensearch-spark_2.12 when run spark application, for example,
6465
```
65-
bin/spark-shell --packages "org.opensearch:opensearch-spark-ppl_2.12:0.2.0-SNAPSHOT"
66+
bin/spark-shell --packages "org.opensearch:opensearch-spark-ppl_2.12:0.3.0-SNAPSHOT"
6667
```
6768

6869
## Code of Conduct

build.sbt

+1-1
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,7 @@ lazy val opensearchVersion = "2.6.0"
1010

1111
ThisBuild / organization := "org.opensearch"
1212

13-
ThisBuild / version := "0.2.0-SNAPSHOT"
13+
ThisBuild / version := "0.3.0-SNAPSHOT"
1414

1515
ThisBuild / scalaVersion := scala212
1616

docs/PPL-on-Spark.md

+2-2
Original file line numberDiff line numberDiff line change
@@ -34,7 +34,7 @@ sbt clean sparkPPLCosmetic/publishM2
3434
```
3535
then add org.opensearch:opensearch-spark_2.12 when run spark application, for example,
3636
```
37-
bin/spark-shell --packages "org.opensearch:opensearch-spark-ppl_2.12:0.2.0-SNAPSHOT"
37+
bin/spark-shell --packages "org.opensearch:opensearch-spark-ppl_2.12:0.3.0-SNAPSHOT"
3838
```
3939

4040
### PPL Extension Usage
@@ -46,7 +46,7 @@ spark-sql --conf "spark.sql.extensions=org.opensearch.flint.FlintPPLSparkExtensi
4646
```
4747

4848
### Running With both Flint & PPL Extensions
49-
In order to make use of both flint and ppl extension, one can simply add both jars (`org.opensearch:opensearch-spark-ppl_2.12:0.2.0-SNAPSHOT`,`org.opensearch:opensearch-spark_2.12:0.2.0-SNAPSHOT`) to the cluster's
49+
In order to make use of both flint and ppl extension, one can simply add both jars (`org.opensearch:opensearch-spark-ppl_2.12:0.3.0-SNAPSHOT`,`org.opensearch:opensearch-spark_2.12:0.3.0-SNAPSHOT`) to the cluster's
5050
classpath.
5151

5252
Next need to configure both extensions :

docs/index.md

+3-3
Original file line numberDiff line numberDiff line change
@@ -54,7 +54,7 @@ Currently, Flint metadata is only static configuration without version control a
5454

5555
```json
5656
{
57-
"version": "0.2.0",
57+
"version": "0.3.0",
5858
"name": "...",
5959
"kind": "skipping",
6060
"source": "...",
@@ -570,7 +570,7 @@ For now, only single or conjunct conditions (conditions connected by AND) in WHE
570570
### AWS EMR Spark Integration - Using execution role
571571
Flint use [DefaultAWSCredentialsProviderChain](https://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/auth/DefaultAWSCredentialsProviderChain.html). When running in EMR Spark, Flint use executionRole credentials
572572
```
573-
--conf spark.jars.packages=org.opensearch:opensearch-spark-standalone_2.12:0.2.0-SNAPSHOT \
573+
--conf spark.jars.packages=org.opensearch:opensearch-spark-standalone_2.12:0.3.0-SNAPSHOT \
574574
--conf spark.jars.repositories=https://aws.oss.sonatype.org/content/repositories/snapshots \
575575
--conf spark.emr-serverless.driverEnv.JAVA_HOME=/usr/lib/jvm/java-17-amazon-corretto.x86_64 \
576576
--conf spark.executorEnv.JAVA_HOME=/usr/lib/jvm/java-17-amazon-corretto.x86_64 \
@@ -612,7 +612,7 @@ Flint use [DefaultAWSCredentialsProviderChain](https://docs.aws.amazon.com/AWSJa
612612
```
613613
3. Set the spark.datasource.flint.customAWSCredentialsProvider property with value as com.amazonaws.emr.AssumeRoleAWSCredentialsProvider. Set the environment variable ASSUME_ROLE_CREDENTIALS_ROLE_ARN with the ARN value of CrossAccountRoleB.
614614
```
615-
--conf spark.jars.packages=org.opensearch:opensearch-spark-standalone_2.12:0.2.0-SNAPSHOT \
615+
--conf spark.jars.packages=org.opensearch:opensearch-spark-standalone_2.12:0.3.0-SNAPSHOT \
616616
--conf spark.jars.repositories=https://aws.oss.sonatype.org/content/repositories/snapshots \
617617
--conf spark.emr-serverless.driverEnv.JAVA_HOME=/usr/lib/jvm/java-17-amazon-corretto.x86_64 \
618618
--conf spark.executorEnv.JAVA_HOME=/usr/lib/jvm/java-17-amazon-corretto.x86_64 \

flint-core/src/main/scala/org/opensearch/flint/core/FlintVersion.scala

+2-1
Original file line numberDiff line numberDiff line change
@@ -16,6 +16,7 @@ case class FlintVersion(version: String) {
1616
object FlintVersion {
1717
val V_0_1_0: FlintVersion = FlintVersion("0.1.0")
1818
val V_0_2_0: FlintVersion = FlintVersion("0.2.0")
19+
val V_0_3_0: FlintVersion = FlintVersion("0.3.0")
1920

20-
def current(): FlintVersion = V_0_2_0
21+
def current(): FlintVersion = V_0_3_0
2122
}

0 commit comments

Comments
 (0)