Use MetopDatasets as prototype for loading matrix-like data

[asinghvi17]

It would be very cool to get https://github.com/eumetsat/MetopDatasets.jl as a backend for Rasters. It also looks like all the metopdatasets stuff has matrix lookups, so it would be a good initial target (that still is a CDM dataset) to load matrix lookups.

@lupemba would you be interested in this? Methods like zonal, rasterize and cellarea wouldn't work yet since they will need custom implementations for matrix lookups, but at least we'd have a data source to test off whenever we do those implementations.

[rafaqz]

Yes, great idea. We can at least make Makie plots and other nice things work.

[lupemba]

Yes that sound like a great idea. One of the ideas behind MetopDatasets.jl is too make it compatible with other julia packages.

Is there anything I can do in MetopDatasets.jl to make this easier. I was thinking about extending the CommonDataModel.coord method to give a uniform way of getting latitude and longitude.

Feel free to ask question and raise issues on the package.

[rafaqz]

Should be fairly easy to add to Rasters, adding new CDM backends is down to a few lines now without anything on your part. We don't use the coord stuff.

Handling reading the matrix lookups is the bit we need to write.

[lupemba]

You might run into some challenge with the data. ASCAT SZR and SZO are probably the easiest but there is a swath on both side of the satellite. If you just treat the two swaths as one raster you are probably going to get weird results between the two swaths. https://eumetsat.github.io/MetopDatasets.jl/dev/ASCAT/#Global-plot-of-SZR-or-SZO

I think the IASI data is more challenging because it has many dimensions. This is the variable that stores the coordinates in the IASI L1C.

ggeosondloc (2 × 4 × 30 × 765)
  Datatype:    Float64 (Int32)
  Dimensions:  lon_lat × sounder_pixel × xtrack × atrack
  Attributes:
   description          = Location of pixel centre in geodetic coordinates (long, lat) for each sounder pixel
   missing_value        = Int32[-2147483648]
   scale_factor         = 1.0e-6

Note that latitude and longitude are stored together. The is a sounder_pixel dimension because the observations comes in groups of 4. The xtrack is the dimension across the satellite ground track. Finally atrack is the dimension along the satellite track which also corresponds to the time dimensions since the satellite is moving along the track while it is recording data.

[rafaqz]

Ok wow. This is an amazing test case 😂

(We can probably load anything if we can load this!)