How do I best handle a state that can change in many subtle ways?

[henrikjacobsenfys]

In my SampleModel I will generate a list of ComponentCollection (self._component_collections, generated by _generate_component_collections(self)); one for each Q that the user gives. The list is generated by making copies from a "master" ComponentCollection (self._components) and a list of DiffusionModels (which itself generates a list of ComponentCollection).

How do I make sure that self._component_collections stays up to date?

The simplest solution is of course to call self._generate_component_collections() any time someone asks for the list of ComponentCollections, but that seems very wasteful (I'll generate tens to hundreds of new unique Parameters every time). It's especially bad if I'm fitting something, since each iteration would generate new collections.
It's perhaps straightforward enough to check if users update self._Q, self._components and self._diffusion_models and update self._component_collections accordingly. Question 1: What is the best way to actually do this?

There's a deeper problem as well.
Question 2: What do I do if a user makes changes to e.g. the underlying ComponentCollection? E.g.

components=ComponentCollection()
components.add_component(Gaussian(unique_name='Gaussian',area=1.0))
sample_model=SampleModel(components=components,Q=[1.0, 2.0])

y1=sample_model.evaluate(x=np.linspace(-1,1,101))

sample_model.components[0].area=2.0

y2=sample_model.evaluate(x=np.linspace(-1,1,101))

When creating the SampleModel I generate the components corresponding to the input Q and models using self._generate_component_collections(). the evaluate method then uses the generated ComponentCollection's evaluate at the input x. Therefore, y1 will be a list of length 2, each element will be 101 values corresponding to the Gaussian evaluated at Q=1.0 and Q=2.0, respectively. This Gaussian will have area 1.0.
Ideally, I would want y2 to be a similar list, but with a Gaussian with area 2, since I updated the value. But how do I make that happen without calling _generate_component_collections inside the evaluate method? I can implement checks that the user changes the components using the methods in SampleModel, but I'm not sure if/how to check if the user changes e.g. a Parameter in collection.

I hope I made this somewhat clear.

[henrikjacobsenfys]

I suppose one solution to part 2 is that I treat the "master" model as an initial guess and make that very explicit in naming it and examples. Then I can also introduce explicit methods (with examples) to change the parameters of the generated ComponentCollections. Then there's of course the question of how to save the changes a user might make to a collection if they then e.g. add or change the diffusionmodel, which might update the collection.

I could of course have two internal lists, one for the diffusion models and one for other models. Hmm..

[henrikjacobsenfys]

Perhaps I need to add a Parent or similar attribute to the ComponentCollection or more likely to the ModelComponents to keep track of who made them. This may make it easier to know when to make changes. Hmm

[rozyczko]

Why not use the Parameter's callback mechanism to silently update the state? This is what it has been created for.
This will, however, only work on value change. If you need state update on variance, error, convert unit, min/max, then you need a proper observer pattern implementation with _attach_observer (also, helpfully present in corelib)

[rozyczko]

Something maybe like

class SampleModel(ModelBase):
    def __init__(self, components, Q, ...):
       ......
        # Cache tracking
        self._dirty = True
        self._attach_callbacks_to_master_parameters()
    
    def _attach_callbacks_to_master_parameters(self) -> None:
        for param in self._components.get_all_variables():
            if isinstance(param, Parameter):
                callback = property(
                    fset=lambda value, self=self: self._on_parameter_changed(value),
                )
                param._callback = callback
    
    def _on_parameter_changed(self, new_value) -> None:
        """Called when any master parameter value changes."""
        self._dirty = True

and then regenerate the collection inside the eval loop only when _dirty is set

[henrikjacobsenfys]

Interesting, I'll have a look. However, I'm not also second-guessing myself a bit, perhaps I don't want to silently change parameters all the time..