diff --git a/README.md b/README.md index 299e1c94e..2d5638a97 100644 --- a/README.md +++ b/README.md @@ -39,7 +39,7 @@ 2.3.* - v0.10.0 + v0.11.0 2.4.0 diff --git a/benchmark/scala/pom.xml b/benchmark/scala/pom.xml index dcedfa5a9..56b8dc1ea 100644 --- a/benchmark/scala/pom.xml +++ b/benchmark/scala/pom.xml @@ -3,7 +3,7 @@ 4.0.0 com.microsoft.spark microsoft-spark-benchmark - 0.10.0 + 0.11.0 2019 UTF-8 diff --git a/docs/release-notes/0.11/release-0.11.md b/docs/release-notes/0.11/release-0.11.md new file mode 100644 index 000000000..e5fd3efa9 --- /dev/null +++ b/docs/release-notes/0.11/release-0.11.md @@ -0,0 +1,101 @@ +# .NET for Apache Spark 0.11 Release Notes + +### New Features and Improvements + +* Ability to pass and return [corefxlab](https://github.com/dotnet/corefxlab) DataFrames to UDF APIs ([#277](https://github.com/dotnet/spark/pull/277)) +* Support for ML TF-IDF (Term frequency-inverse document frequency) feature vectorization method ([#394](https://github.com/dotnet/spark/pull/394)) +* Support for TimestampType in `DataFrame.Collect()`, `CreateDataFrame` and UDFs ([#428](https://github.com/dotnet/spark/pull/428)) +* Support for Broadcast Variables ([#414](https://github.com/dotnet/spark/pull/414)) +* Support for ML feature Word2Vec ([#491](https://github.com/dotnet/spark/pull/491)) +* Streamline logging when there is a failure ([#439](https://github.com/dotnet/spark/pull/439)) + + +### Breaking Changes + +* `SparkSession.Catalog` is changed from a method to a property ([#508](https://github.com/dotnet/spark/pull/508)) + +### Compatibility + +#### Backward compatibility + +The following table describes the oldest version of the worker that the current version is compatible with, along with new features that are incompatible with the worker. + + + + + + + + + + + + + + + + + + + + + + + +
Oldest compatible Microsoft.Spark.Worker versionIncompatible features
v0.9.0DataFrame with Grouped Map UDF (#277)
DataFrame with Vector UDF (#277)
Support for Broadcast Variables (#414)
Support for TimestampType (#428)
+ +#### Forward compatibility + +The following table describes the oldest version of .NET for Apache Spark release that the current worker is compatible with. + + + + + + + + + + + + +
Oldest compatible .NET for Apache Spark release version
v0.9.0
+ +### Supported Spark Versions + +The following table outlines the supported Spark versions along with the microsoft-spark JAR to use with: + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Spark Versionmicrosoft-spark JAR
2.3.*microsoft-spark-2.3.x-0.11.0.jar
2.4.0microsoft-spark-2.4.x-0.11.0.jar
2.4.1
2.4.3
2.4.4
2.4.5
2.4.2Not supported
diff --git a/eng/Versions.props b/eng/Versions.props index f506459e1..18d664dda 100644 --- a/eng/Versions.props +++ b/eng/Versions.props @@ -1,7 +1,7 @@ - 0.10.0 + 0.11.0 prerelease $(RestoreSources); diff --git a/src/scala/pom.xml b/src/scala/pom.xml index ddcd7645f..34ee5c338 100644 --- a/src/scala/pom.xml +++ b/src/scala/pom.xml @@ -7,7 +7,7 @@ ${microsoft-spark.version} UTF-8 - 0.10.0 + 0.11.0