Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

valid_range attribute is unsafely cast on NETCDF4_CLASSIC save #6178

Open
jrackham-mo opened this issue Oct 16, 2024 · 0 comments
Open

valid_range attribute is unsafely cast on NETCDF4_CLASSIC save #6178

jrackham-mo opened this issue Oct 16, 2024 · 0 comments

Comments

@jrackham-mo
Copy link
Contributor

🐛 Bug Report

When saving a cube to a netCDF file using the NETCDF4_CLASSIC format, the valid_range attribute is unsafely cast.

How To Reproduce

Steps to reproduce the behaviour:

  1. Create a cube with a 64-bit int valid_range attribute with a value greater than the maximum for a 32-bit int
  2. Save the cube as netCDF using format NETCDF4_CLASSIC
  3. Inspect the saved file, valid_range has been unsafely cast
import iris.fileformats
import iris.cube
import numpy as np
import netCDF4

if __name__ == "__main__":
    data = np.arange(5, dtype="int64")
    cube = iris.cube.Cube(data)
    cube.attributes["valid_range"] = [0, np.iinfo(cube.dtype).max]
    print(cube.attributes)
    iris.fileformats.netcdf.save(cube, "foo.nc", netcdf_format="NETCDF4_CLASSIC")

    foo = netCDF4.Dataset("foo.nc")
    print(foo)
    foo.close()
{'valid_range': [0, 9223372036854775807]}
...
<class 'netCDF4._netCDF4.Dataset'>
root group (NETCDF4_CLASSIC data model, file format HDF5):
    valid_range: [ 0 -1]
    Conventions: CF-1.7
    dimensions(sizes): dim0(5)
    variables(dimensions): int32 unknown(dim0)
    groups: 

Notice that the data has been cast to int32 as expected, but the valid_range has been corrupted.

Expected behaviour

An error should be raised to inform that the valid_range cannot be safely cast to an allowed netCDF classic type. This would be similar to the current behaviour when a too-large value appears in the data:

import iris.fileformats
import iris.cube
import numpy as np

if __name__ == "__main__":
    data = np.arange(5, dtype="int64")
    cube = iris.cube.Cube(data)
    cube.data[-1] = np.iinfo(cube.dtype).max
    iris.fileformats.netcdf.save(cube, "foo.nc", netcdf_format="NETCDF4_CLASSIC")
ValueError: The data type of cube <iris 'Cube' of unknown / (unknown) (-- : 5)> is not supported by NETCDF4_CLASSIC and its values cannot be safely cast to a supported integer type.

Screenshots

Environment

  • OS & Version: RHEL7
  • Iris Version: 3.10.0
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: No status
Development

No branches or pull requests

2 participants