Description
Currently, fx_file
has the call signature (filename) -> (filename)
. Thus, files need to be copied if they need to be modified (overwriting input files is a very bad idea), which is very expensive. Usually, the netCDF4
library is used to perform these kind of modifications.
A possible alternative that has often come up is xarray. At the moment, there is a method DataArray.to_iris()
which allows converting a DataArray
into a Cube
(no idea how efficient this is, though). In addition, the iris devs are working on an improved xarray-iris bridge.
Thus, it would be very nice to allow an efficient usage of xarray in fix_file
. My question is now: how would that look like in practice? Possible solutions I could think of are:
- Allow
fx_file
to return xarray objects (currently onlyDataArray
?) andload
to read these kind of objects. - Allow
fx_file
to return cubes andload
to read cubes. - Rewrite
fx_file
in such a way that it is a callback inload
(with one of the call signatures above).
The reason I am asking this now is that I want to get started with reading the ERA5 GRIB files that are available on Levante (see #1991). However, iris-grib cannot open them. On the other hand, xarray (in combination with ECMWF's cfgrib) works perfectly fine. Thus, an option to include xarray into our pipeline would be super helpful for me.
@ESMValGroup/technical-lead-development-team opinions?