Skip to content

Releases: google/xarray-beam

0.7.0: Add a warning when using xarray_beam.ChunksToZarr with no template

18 Feb 21:45
Compare
Choose a tag to compare
If this works well at some point in the future (e.g., using zarr v3), we can always remove the warning. But for now, this is better than letting users run directly into the race condition in https://github.com/google/xarray-beam/issues/85

I've also removed all suggestions from the docs of omitting a `template`.

PiperOrigin-RevId: 728341864

release version 0.6.3

01 Dec 18:58
Compare
Choose a tag to compare
PiperOrigin-RevId: 587056902

0.6.2: Fix split_variables with different dimensions per variable

30 May 15:17
Compare
Choose a tag to compare

0.6.1: [xarray-beam] fix docs & bug in make_template

28 May 17:46
Compare
Choose a tag to compare
Reusing dask.array names for arrays with different shapes was the indirect cause of the error on our docs pages.

I've also updated the build dependencies for our docs to fix other build issues, and added a Python 3.11 run of our unit tests.

Fixes https://github.com/google/xarray-beam/issues/76

PiperOrigin-RevId: 536022594

0.6.0: Use multi-stage rechunking.

28 Mar 18:00
Compare
Choose a tag to compare
As described in https://github.com/pangeo-data/rechunker/pull/89,
this can yield significant performance benefits for rechunking large
arrays.

PiperOrigin-RevId: 518325665

0.5.1: [xarray-beam] update a missing version number

18 Nov 23:32
Compare
Choose a tag to compare

0.4.1: Remove pyformat comments

27 Sep 21:23
Compare
Choose a tag to compare
PiperOrigin-RevId: 477271219

Release Xarray-Beam 0.4.0

03 Sep 04:41
Compare
Choose a tag to compare
Support datasets with differently chunked variables in DatasetToChunks

There are two major internal changes:
1. Key objects from DatasetToChunks now can include different dimensions for different variables when using split_vars=True. This makes it easier to handle large datasets with many variables and different chunking per variable.
2. Inputs inside the DatasetToChunks pipeline can now be sharded across many tasks. This is important for scalability to large datasets, especially with this chagne because the above refactor increases the number of inputs by the number of variables when split_vars=True. Otherwise, we can run into performance issues on the machine launching the pipeline when the number of inputs goes into the millions (e.g., slow speed, out of memory).

See the new integration test for a concrete use-case, resembling real model output.

Also revise the warning message in the README to be a bit friendlier.

Fixes https://github.com/google/xarray-beam/issues/43

PiperOrigin-RevId: 471948735

Release Xarray-Beam 0.3.1

01 Jun 16:59
Compare
Choose a tag to compare
PiperOrigin-RevId: 452320840

Release Xarary-Beam version 0.3.0

25 May 17:33
Compare
Choose a tag to compare
There are some minor bug fixes here that would be good to get out!

PiperOrigin-RevId: 450957392