When to use "deep copy"? (Doc string is over my head)

[michaelaye]

Dask delayed best practices advises to not mutate functional inputs.
As I'm pushing small xarray.DataArrays around that I change in the computation, I now understand I should copy them.
But I cannot find any advise on if that should be a deep copy or not?
The docstring for the deep copy option is frankly over my head. I'm just a poor scientist that for a good reason changed from C++ to Python all those years ago, I don't want to manage memory, I want to look at my data. :) ( I figured in the discussion part of xarray I could add such a comment without sounding obnoxious, I wouldn't do so in an issue.
Any advise on this appreciated?

[max-sixty]

Others can give more precise definitions but my mental model:

• .deepcopy copies everything that the object references. It shares nothing with the existing object.
• .copy only copies the container. Anything that the existing object referenced will still be referenced by the new object.

One way to see this is to try copy and deepcopy-ing a dataset and see how these two respective operations on the copy are affected:

• Assign an attrs
• Mutate the underlying array; i.e. ds["lat"][0] = 1

[max-sixty]

Thanks @keewis , my comment re "assign an attr" was incorrect, appreciate you correcting me!

As you say — the important difference is whether the underlying data is copied. "Mutate the underlying array; i.e. ds["lat"][0] = 1" will show that.

[keewis]

actually, you were right about attrs also being affected by this:

attrs copy

In [3]: import xarray as xr
   ...: 
   ...: ds = xr.Dataset({"a": ("x", ["a", "b", "c"])}, attrs={"attr": ["a", "b"]})
   ...: shallow_copy = ds.copy(deep=False)
   ...: deep_copy = ds.copy(deep=True)
   ...: 
   ...: print("original dataset:", ds, sep="\n")
   ...: 
   ...: ds.attrs["attr"].append("c")
   ...: ds.a[0] = "z"
   ...: 
   ...: print("modified dataset:", ds, sep="\n")
   ...: print("shallow copy:", shallow_copy, sep="\n")
   ...: print("deep copy:", deep_copy, sep="\n")
original dataset:
<xarray.Dataset>
Dimensions:  (x: 3)
Dimensions without coordinates: x
Data variables:
    a        (x) <U1 'a' 'b' 'c'
Attributes:
    attr:     ['a', 'b']
modified dataset:
<xarray.Dataset>
Dimensions:  (x: 3)
Dimensions without coordinates: x
Data variables:
    a        (x) <U1 'z' 'b' 'c'
Attributes:
    attr:     ['a', 'b', 'c']
shallow copy:
<xarray.Dataset>
Dimensions:  (x: 3)
Dimensions without coordinates: x
Data variables:
    a        (x) <U1 'z' 'b' 'c'
Attributes:
    attr:     ['a', 'b', 'c']
deep copy:
<xarray.Dataset>
Dimensions:  (x: 3)
Dimensions without coordinates: x
Data variables:
    a        (x) <U1 'a' 'b' 'c'
Attributes:
    attr:     ['a', 'b']

[max-sixty]

You're insufficiently critical of me @keewis ! 😄

In order to see the difference between copy & deepcopy in attrs, we have to mutate the value of an attr. Adding an attr doesn't show any difference.

In that way, a Dataset is not the same as a python object like a list or dict. That's the element I think you corrected me on.

Do you agree?

[keewis]

my understanding of .copy is that it applies copy.copy / copy.deepcopy to whatever it is wrapping, so what exactly happens is determined by their implementation of __copy__ / __deepcopy__. dict and ndarray happen to reference the data if they are shallow-copied, but other types (i.e. duckarray objects) might do something different.

[dcherian]

(we should add this to the docstring)