HDF4 support

[adamjstewart]

I'm interested in reading MODIS imagery, which is distributed as HDF4 files. It looks like xarray used to support PyNIO for reading HDF4 files but that dependency was removed recently. Is there any interest in resurrecting HDF4 support, either through PyNIO or a different dependency?

[kmuehlbauer]

@adamjstewart AFAIK HDF4 based files can be read via engine='rasterio' in the call to open_dataset, given the needed libraries (rasterio, rioxarray) are installed.

[adamjstewart]

Only if rasterio is built with GDAL that is built with HDF4 support, which is not the case for the rasterio wheels from PyPI.

[kmuehlbauer]

Point taken, that's the downside. You could also go the way via the containers provided by GDAL or the conda-forge ecosystem.

IIRC there is also the option to read some HDF4 files via netcdf4-python and netcdf-c (if this enabled).

Is there any interest in resurrecting HDF4 support, either through PyNIO or a different dependency?

Maybe others have or had the same issues reading HDF4. Would be good if those would chime in, how they solved it in their environments.

[keewis]

pynio has been abandoned, so we can't use that. Not sure about built-in support, but anyone is free to create a new xarray backend for HDF4 and distribute it in a separate package, so that's definitely an option. Otherwise kerchunk has an experimental HDF4 reader, so it might also be possible to read HDF4 by opening as a virtual zarr file.

[adamjstewart]

https://pypi.org/project/pyhdf/ seems to be the best maintained option. For now, I will try to directly read with pyhdf and create an in-memory xarray Dataset. Not sure how performant that is for large files though.