Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
91 changes: 91 additions & 0 deletions docs/archive.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,97 @@ nav_order: 15
---
Below are archived releases for RAPIDS Accelerator for Apache Spark.

## Release v25.10.0
### Hardware Requirements:

The plugin is designed to work on NVIDIA Volta, Turing, Ampere, Ada Lovelace, Hopper and Blackwell generation datacenter GPUs. The plugin jar is tested on the following GPUs:

GPU Models: NVIDIA V100, T4, A10, A100, L4, H100 and B100 GPUs

### Software Requirements:

OS: Spark RAPIDS is compatible with any Linux distribution with glibc >= 2.28 (Please check ldd --version output). glibc 2.28 was released August 1, 2018.
Tested on Ubuntu 22.04, Ubuntu 24.04, Rocky Linux 8 and Rocky Linux 9

NVIDIA Driver*: R525+

Runtime:
Scala 2.12, 2.13
Python, Java Virtual Machine (JVM) compatible with your spark-version.

* Check the Spark documentation for Python and Java version compatibility with your specific
Spark version. For instance, visit `https://spark.apache.org/docs/3.4.1` for Spark 3.4.1.

Supported Spark versions:
Apache Spark 3.2.0, 3.2.1, 3.2.2, 3.2.3, 3.2.4
Apache Spark 3.3.0, 3.3.1, 3.3.2, 3.3.3, 3.3.4
Apache Spark 3.4.0, 3.4.1, 3.4.2, 3.4.3, 3.4.4
Apache Spark 3.5.0, 3.5.1, 3.5.2, 3.5.3, 3.5.4, 3.5.5, 3.5.6
Apache Spark 4.0.0, 4.0.1

Supported Databricks runtime versions for Azure and AWS:
Databricks 12.2 ML LTS (GPU, Scala 2.12, Spark 3.3.2)
Databricks 13.3 ML LTS (GPU, Scala 2.12, Spark 3.4.1)
Databricks 14.3 ML LTS (GPU, Scala 2.12, Spark 3.5.0)

Supported Dataproc versions (Debian/Ubuntu/Rocky):
GCP Dataproc 2.1
GCP Dataproc 2.2
GCP Dataproc 2.3

Supported Dataproc Serverless versions:
Spark runtime 1.1 LTS
Spark runtime 1.2
Spark runtime 2.0
Spark runtime 2.1
Spark runtime 2.2

*Some hardware may have a minimum driver version greater than R470. Check the GPU spec sheet
for your hardware's minimum driver version.

*For Cloudera and EMR support, please refer to the
[Distributions](https://docs.nvidia.com/spark-rapids/user-guide/latest/faq.html#which-distributions-are-supported) section of the FAQ.

### RAPIDS Accelerator's Support Policy for Apache Spark
The RAPIDS Accelerator maintains support for Apache Spark versions available for download from [Apache Spark](https://spark.apache.org/downloads.html)

### Download RAPIDS Accelerator for Apache Spark v25.10.0

| Processor | Scala Version | Download Jar | Download Signature | Download From Maven |
|-----------|---------------|--------------|--------------------|---------------------|
| x86_64 | Scala 2.12 | [RAPIDS Accelerator v25.10.0](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.12/25.10.0/rapids-4-spark_2.12-25.10.0.jar) | [Signature](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.12/25.10.0/rapids-4-spark_2.12-25.10.0.jar.asc) | <pre>&lt;dependency&gt;<br/> &lt;groupId&gt;com.nvidia&lt;/groupId&gt;<br/> &lt;artifactId&gt;rapids-4-spark_2.12&lt;/artifactId&gt;<br/> &lt;version&gt;25.10.0&lt;/version&gt;<br/>&lt;/dependency&gt;</pre> |
| x86_64 | Scala 2.13 | [RAPIDS Accelerator v25.10.0](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.13/25.10.0/rapids-4-spark_2.13-25.10.0.jar) | [Signature](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.13/25.10.0/rapids-4-spark_2.13-25.10.0.jar.asc) | <pre>&lt;dependency&gt;<br/> &lt;groupId&gt;com.nvidia&lt;/groupId&gt;<br/> &lt;artifactId&gt;rapids-4-spark_2.13&lt;/artifactId&gt;<br/> &lt;version&gt;25.10.0&lt;/version&gt;<br/>&lt;/dependency&gt;</pre> |
| arm64 | Scala 2.12 | [RAPIDS Accelerator v25.10.0](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.12/25.10.0/rapids-4-spark_2.12-25.10.0-cuda12-arm64.jar) | [Signature](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.12/25.10.0/rapids-4-spark_2.12-25.10.0-cuda12-arm64.jar.asc) | <pre>&lt;dependency&gt;<br/> &lt;groupId&gt;com.nvidia&lt;/groupId&gt;<br/> &lt;artifactId&gt;rapids-4-spark_2.12&lt;/artifactId&gt;<br/> &lt;version&gt;25.10.0&lt;/version&gt;<br/> &lt;classifier&gt;cuda12-arm64&lt;/classifier&gt;<br/>&lt;/dependency&gt;</pre> |
| arm64 | Scala 2.13 | [RAPIDS Accelerator v25.10.0](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.13/25.10.0/rapids-4-spark_2.13-25.10.0-cuda12-arm64.jar) | [Signature](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.13/25.10.0/rapids-4-spark_2.13-25.10.0-cuda12-arm64.jar.asc) | <pre>&lt;dependency&gt;<br/> &lt;groupId&gt;com.nvidia&lt;/groupId&gt;<br/> &lt;artifactId&gt;rapids-4-spark_2.13&lt;/artifactId&gt;<br/> &lt;version&gt;25.10.0&lt;/version&gt;<br/> &lt;classifier&gt;cuda12-arm64&lt;/classifier&gt;<br/>&lt;/dependency&gt;</pre> |

This package is built against CUDA 12.9. It is tested on V100, T4, A10, A100, L4, H100 and GB100 GPUs with
CUDA 12.9.

### Verify signature
* Download the [PUB_KEY](https://keys.openpgp.org/search?q=sw-spark@nvidia.com).
* Import the public key: `gpg --import PUB_KEY`
* Verify the signature for Scala 2.12 jar:
`gpg --verify rapids-4-spark_2.12-25.10.0.jar.asc rapids-4-spark_2.12-25.10.0.jar`
* Verify the signature for Scala 2.13 jar:
`gpg --verify rapids-4-spark_2.13-25.10.0.jar.asc rapids-4-spark_2.13-25.10.0.jar`

The output of signature verify:

gpg: Good signature from "NVIDIA Spark (For the signature of spark-rapids release jars) <sw-spark@nvidia.com>"

### Release Notes
* Delta Lake liquid clustering read/write/optimize support
* Delta Lake optimize support
* Delta Lake deletion vector support with two caveats: need to set `useMetadataRowIndex=false` and deletion vector support will fall back to the CPU when using the coalescing file reader (these limitations to be removed in a future release)
* Iceberg insert operations support and improved write job statistics tracking
* Improved performance for stddev and variance operations in hash based group by aggregations
* Support for uuid
* Added Spark 4.0.1 support
* Added CUDA 13 support, in addition to CUDA 12 support.

Note: There is a known issue in the 25.10.0 release when decompressing gzip files on H100 GPUs.
Please find more details in [issue-16661](https://github.com/rapidsai/cudf/issues/16661).

## Release v25.08.0
### Hardware Requirements:

Expand Down
34 changes: 16 additions & 18 deletions docs/download.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ cuDF jar, that is either preinstalled in the Spark classpath on all nodes or sub
that uses the RAPIDS Accelerator For Apache Spark. See the [getting-started
guide](https://docs.nvidia.com/spark-rapids/user-guide/latest/getting-started/overview.html) for more details.

## Release v25.10.0
## Release v25.12.0
### Hardware Requirements:

The plugin is designed to work on NVIDIA Volta, Turing, Ampere, Ada Lovelace, Hopper and Blackwell generation datacenter GPUs. The plugin jar is tested on the following GPUs:
Expand Down Expand Up @@ -72,14 +72,14 @@ for your hardware's minimum driver version.
### RAPIDS Accelerator's Support Policy for Apache Spark
The RAPIDS Accelerator maintains support for Apache Spark versions available for download from [Apache Spark](https://spark.apache.org/downloads.html)

### Download RAPIDS Accelerator for Apache Spark v25.10.0
### Download RAPIDS Accelerator for Apache Spark v25.12.0

| Processor | Scala Version | Download Jar | Download Signature | Download From Maven |
|-----------|---------------|--------------|--------------------|---------------------|
| x86_64 | Scala 2.12 | [RAPIDS Accelerator v25.10.0](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.12/25.10.0/rapids-4-spark_2.12-25.10.0.jar) | [Signature](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.12/25.10.0/rapids-4-spark_2.12-25.10.0.jar.asc) | <pre>&lt;dependency&gt;<br/> &lt;groupId&gt;com.nvidia&lt;/groupId&gt;<br/> &lt;artifactId&gt;rapids-4-spark_2.12&lt;/artifactId&gt;<br/> &lt;version&gt;25.10.0&lt;/version&gt;<br/>&lt;/dependency&gt;</pre> |
| x86_64 | Scala 2.13 | [RAPIDS Accelerator v25.10.0](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.13/25.10.0/rapids-4-spark_2.13-25.10.0.jar) | [Signature](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.13/25.10.0/rapids-4-spark_2.13-25.10.0.jar.asc) | <pre>&lt;dependency&gt;<br/> &lt;groupId&gt;com.nvidia&lt;/groupId&gt;<br/> &lt;artifactId&gt;rapids-4-spark_2.13&lt;/artifactId&gt;<br/> &lt;version&gt;25.10.0&lt;/version&gt;<br/>&lt;/dependency&gt;</pre> |
| arm64 | Scala 2.12 | [RAPIDS Accelerator v25.10.0](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.12/25.10.0/rapids-4-spark_2.12-25.10.0-cuda12-arm64.jar) | [Signature](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.12/25.10.0/rapids-4-spark_2.12-25.10.0-cuda12-arm64.jar.asc) | <pre>&lt;dependency&gt;<br/> &lt;groupId&gt;com.nvidia&lt;/groupId&gt;<br/> &lt;artifactId&gt;rapids-4-spark_2.12&lt;/artifactId&gt;<br/> &lt;version&gt;25.10.0&lt;/version&gt;<br/> &lt;classifier&gt;cuda12-arm64&lt;/classifier&gt;<br/>&lt;/dependency&gt;</pre> |
| arm64 | Scala 2.13 | [RAPIDS Accelerator v25.10.0](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.13/25.10.0/rapids-4-spark_2.13-25.10.0-cuda12-arm64.jar) | [Signature](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.13/25.10.0/rapids-4-spark_2.13-25.10.0-cuda12-arm64.jar.asc) | <pre>&lt;dependency&gt;<br/> &lt;groupId&gt;com.nvidia&lt;/groupId&gt;<br/> &lt;artifactId&gt;rapids-4-spark_2.13&lt;/artifactId&gt;<br/> &lt;version&gt;25.10.0&lt;/version&gt;<br/> &lt;classifier&gt;cuda12-arm64&lt;/classifier&gt;<br/>&lt;/dependency&gt;</pre> |
| x86_64 | Scala 2.12 | [RAPIDS Accelerator v25.12.0](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.12/25.12.0/rapids-4-spark_2.12-25.12.0.jar) | [Signature](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.12/25.12.0/rapids-4-spark_2.12-25.12.0.jar.asc) | <pre>&lt;dependency&gt;<br/> &lt;groupId&gt;com.nvidia&lt;/groupId&gt;<br/> &lt;artifactId&gt;rapids-4-spark_2.12&lt;/artifactId&gt;<br/> &lt;version&gt;25.12.0&lt;/version&gt;<br/>&lt;/dependency&gt;</pre> |
| x86_64 | Scala 2.13 | [RAPIDS Accelerator v25.12.0](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.13/25.12.0/rapids-4-spark_2.13-25.12.0.jar) | [Signature](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.13/25.12.0/rapids-4-spark_2.13-25.12.0.jar.asc) | <pre>&lt;dependency&gt;<br/> &lt;groupId&gt;com.nvidia&lt;/groupId&gt;<br/> &lt;artifactId&gt;rapids-4-spark_2.13&lt;/artifactId&gt;<br/> &lt;version&gt;25.12.0&lt;/version&gt;<br/>&lt;/dependency&gt;</pre> |
| arm64 | Scala 2.12 | [RAPIDS Accelerator v25.12.0](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.12/25.12.0/rapids-4-spark_2.12-25.12.0-cuda12-arm64.jar) | [Signature](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.12/25.12.0/rapids-4-spark_2.12-25.12.0-cuda12-arm64.jar.asc) | <pre>&lt;dependency&gt;<br/> &lt;groupId&gt;com.nvidia&lt;/groupId&gt;<br/> &lt;artifactId&gt;rapids-4-spark_2.12&lt;/artifactId&gt;<br/> &lt;version&gt;25.12.0&lt;/version&gt;<br/> &lt;classifier&gt;cuda12-arm64&lt;/classifier&gt;<br/>&lt;/dependency&gt;</pre> |
| arm64 | Scala 2.13 | [RAPIDS Accelerator v25.12.0](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.13/25.12.0/rapids-4-spark_2.13-25.12.0-cuda12-arm64.jar) | [Signature](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.13/25.12.0/rapids-4-spark_2.13-25.12.0-cuda12-arm64.jar.asc) | <pre>&lt;dependency&gt;<br/> &lt;groupId&gt;com.nvidia&lt;/groupId&gt;<br/> &lt;artifactId&gt;rapids-4-spark_2.13&lt;/artifactId&gt;<br/> &lt;version&gt;25.12.0&lt;/version&gt;<br/> &lt;classifier&gt;cuda12-arm64&lt;/classifier&gt;<br/>&lt;/dependency&gt;</pre> |

This package is built against CUDA 12.9. It is tested on V100, T4, A10, A100, L4, H100 and GB100 GPUs with
CUDA 12.9.
Expand All @@ -88,25 +88,23 @@ CUDA 12.9.
* Download the [PUB_KEY](https://keys.openpgp.org/search?q=sw-spark@nvidia.com).
* Import the public key: `gpg --import PUB_KEY`
* Verify the signature for Scala 2.12 jar:
`gpg --verify rapids-4-spark_2.12-25.10.0.jar.asc rapids-4-spark_2.12-25.10.0.jar`
`gpg --verify rapids-4-spark_2.12-25.12.0.jar.asc rapids-4-spark_2.12-25.12.0.jar`
* Verify the signature for Scala 2.13 jar:
`gpg --verify rapids-4-spark_2.13-25.10.0.jar.asc rapids-4-spark_2.13-25.10.0.jar`
`gpg --verify rapids-4-spark_2.13-25.12.0.jar.asc rapids-4-spark_2.13-25.12.0.jar`

The output of signature verify:

gpg: Good signature from "NVIDIA Spark (For the signature of spark-rapids release jars) <sw-spark@nvidia.com>"

### Release Notes
* Delta Lake liquid clustering read/write/optimize support
* Delta Lake optimize support
* Delta Lake deletion vector support with two caveats: need to set `useMetadataRowIndex=false` and deletion vector support will fall back to the CPU when using the coalescing file reader (these limitations to be removed in a future release)
* Iceberg insert operations support and improved write job statistics tracking
* Improved performance for stddev and variance operations in hash based group by aggregations
* Support for uuid
* Added Spark 4.0.1 support
* Added CUDA 13 support, in addition to CUDA 12 support.

Note: There is a known issue in the 25.10.0 release when decompressing gzip files on H100 GPUs.
* Iceberg enhancements including DML operations (delete, update, merge) for merge-on-read tables, partition transforms (year/month/day/hour/truncate), and write operations enabled by default.
* Delta Lake clustered tables DML support including update, merge, and delete operations with deletion vector enabled GPU by default.
* Join improvements including support for left-outer joins with no columns, new join strategies with logging and heuristic configurations, and improved gather map ordering.
* CSV support for GBK encoded data.
* Refine GpuTaskMetrics over the spill framework.
* Fix race condition due to premature disk handle exposure.

Note: There is a known issue in the 25.12.0 release when decompressing gzip files on H100 GPUs.
Please find more details in [issue-16661](https://github.com/rapidsai/cudf/issues/16661).

For a detailed list of changes, please refer to the
Expand Down