Autosave support

[jsouter]

Addressing #162
Adding in support for EPICS autosave style periodic saving of PV values and fields to a yaml-based backup file, from which initial values can be loaded on a restart.

[jsouter]

Just tested with all the builder record types, looks like it's not happy with waveform record types (numpy array not serialisable to json), will either have to cast these to lists or try dumping to a yaml file, possibly with custom representers. Or use some sort of pickle-type approach, but probably preferable to be human readable.

[AlexanderWells-diamond]

Looks good overall, but I've raised a few questions about what exactly the API could/should be in various comments.

There's several bits that need doing:

• Tests; at least a couple of system-level tests, then as much unit testing as you feel appropriate. Any test that starts an IOC will need to use the multiprocessing mechanism, as various parts of EPICS won't allow multiple IOC initializations in a single thread so a new one must be used for each IOC. A simple template is probably test_record_wrapper_str in test_records.py.
• Providing example code, either inside the existing docs/examples/example_*.ioc.py file(s) or in a separate new one
• Docs; Probably fine to make a new how-to document explaining how to configure and use autosave.
• A CHANGELOG entry - take a look at previous entries to see the format/syntax.

[jsouter]

I have added support for autosaving non-VAL fields, which is done by calling
builder.aOut("PV-NAME", autosave_fields=["SCAN", "PREC"]) for example. The autosave=True argument is used to determine if we add the VAL field to autosave, which I think is less ugly than requring the use to pass ["VAL"] to autosave_fields. (I note that this all needs to be added to the documentation when it's decided). I have noted however that In type records raise an exception when calling set_field, which gets called when loading from an autosave file.
It could be possible to get around this by changing the setter to use the . accessor for fields, (pv.SCAN = "Passive") when loading, but we would have to make sure this load gets done before LoadDatabase() is called, which would be hard to enforce unless require the user to explicitly call something like a classmethod Autosave.load() before LoadDatabase() and softioc.iocInit().

EDIT: Have realised that alternatively we could force DISP=0 for In type records that require fields to be tracked in autosave.

[AlexanderWells-diamond]

The overall code and structure seems pretty good now. Unfortunately there's a number of bits causing problems, and a handful of tasks that still need doing.

There's a lot of CI test failures, some of which are due to the Pipfile.lock change, and some of them to do with the autosave code.

For the Pipfile changes, there's a comment discussing whether they are needed or not.

For the other CI fails, I found some errors like this in a few places:

   ----------------------------- Captured stderr call -----------------------------
  2024-07-16T09:07:11.552312123 WARN pvxs.tcp.setup Server unable to bind port 5075, falling back to [::]:39597
  Process ForkServerProcess-5:
  Traceback (most recent call last):
    File "/opt/python/cp38-cp38/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap
      self.run()
    File "/opt/python/cp38-cp38/lib/python3.8/multiprocessing/process.py", line 108, in run
      self._target(*self._args, **self._kwargs)
    File "/project/tests/test_record_values.py", line 384, in run_ioc
      builder.LoadDatabase()
    File "/tmp/tmp.ZezuRo2hXU/venv/lib/python3.8/site-packages/softioc/builder.py", line 303, in LoadDatabase
      autosave.load()
    File "/tmp/tmp.ZezuRo2hXU/venv/lib/python3.8/site-packages/softioc/autosave.py", line 70, in load
      Autosave._load()
    File "/tmp/tmp.ZezuRo2hXU/venv/lib/python3.8/site-packages/softioc/autosave.py", line 186, in _load
      cls._backup_sav_file()
    File "/tmp/tmp.ZezuRo2hXU/venv/lib/python3.8/site-packages/softioc/autosave.py", line 117, in _backup_sav_file
      sav_path = cls._get_current_sav_path()
    File "/tmp/tmp.ZezuRo2hXU/venv/lib/python3.8/site-packages/softioc/autosave.py", line 139, in _get_current_sav_path
      return cls.directory / f"{cls.device_name}.{SAV_SUFFIX}"
  TypeError: unsupported operand type(s) for /: 'NoneType' and 'str'

So it seems there are some cases where the autosave code is being initialized with invalid values.

The extra bits of work that still need doing:

• Documentation. api.rst needs extending to include the new keywords added to record initialization, and then we probably also need a new how-to doc to explain autosave and its mechanisms.
• A CHANGELOG entry - copy a previous one and change the info, as the syntax is quite picky!
• Possibly a new example python IOC using autosave (or modifications to an existing one)
• At least one full-system IOC test that actually starts an IOC from a canned autosave file and proves it loads the new values. Feel free to ask me when you get to this task as it is quite complicated to make it work.

[Araneidae]

Looks good, have a few questions to look at.

[Araneidae]

An interesting thought here: builder IN records must have .SCAN set to "I/O Intr" for proper operation (otherwise .set() won't work properly), so using "SCAN" as a saved field isn't the best example! I'd suggest either "EGU" or some of the alarm threshold fields.

[Araneidae]

Why not use __ prefix for local names? This is the normal Python style for member names, and it triggers automatic name mangling to hide the names from outside the class.

[jsouter]

I will spend some time this afternoon looking back over original autosave and see if there's any features of that worth implementing. I'm not sure if there's any meaningful reason to try and implement things like the asVerify and the pass0/pass1 restoring of values before and after record initialization, but there are a few things like manual save restore that could be more relevant. I wonder if using the name autosave implies a sort of feature parity that we don't necessarily want

[AlexanderWells-diamond]

One last feature request: It may well be useful to provide a context manager, that can automatically do Autosave for all PVs declared within it. Possibly also supporting a specific list of fields, something like this:

with Autosave(autosave=True, autosave_fields=["EGU", "PREC"]):
  builder.aOut("MY_RECORD")

And this will autosave MY_RECORD's VAL, EGU, and PREC fields.

[jsouter]

One last feature request: It may well be useful to provide a context manager, that can automatically do Autosave for all PVs declared within it. Possibly also supporting a specific list of fields, something like this:

with Autosave(autosave=True, autosave_fields=["EGU", "PREC"]):
  builder.aOut("MY_RECORD")

And this will autosave MY_RECORD's VAL, EGU, and PREC fields.

I'll push fixes to the above comments soon, I've got lots of things in branches at the moment! The most straightforward way I could think to do this was to define a context manager in builder.py like this

_user_defaults = {}  # for use in DeviceFields context manager

class DeviceFields:
    def __init__(self, **fields):
        self._fields = fields
    
    def __enter__(self):
        global _user_defaults
        _user_defaults = self._fields
    
    def __exit__(self, A, B, C):
        global _user_defaults
        _user_defaults = {}

def _set_user_defaults(fields):
    fields.update(_user_defaults)

and have aOut etc call _set_user_defaults(fields).

This is generic enough that it could be used for any EPICS field like EGU etc, would that be something we want? Or would it be better to rework the Autosave interface to work as a context manager?

with builder.DeviceFields(autosave=True, EGU="mm"):
    builder.aOut("MY-RECORD")

[AlexanderWells-diamond]

I would prefer to see the Autosave class used as the context manager. As we don't have to run any code during the creation of each PV, what we can do is track how many PVs were created during the lifetime of the context manager and then add them in bulk to the Autosave list. Something like this (untested) code:

from softioc.device_core import LookupRecordList

class Autosave:
    def __init__(self, **fields):
        self._fields = fields
    
    def __enter__(self):
       self.before = LookupRecordList()
    
    def __exit__(self, A, B, C):
        after = LookupRecordList()
        # Get diff of the keys in before and after dict, then add them to Autosave

[jsouter]

I would prefer to see the Autosave class used as the context manager. As we don't have to run any code during the creation of each PV, what we can do is track how many PVs were created during the lifetime of the context manager and then add them in bulk to the Autosave list. Something like this (untested) code:

from softioc.device_core import LookupRecordList

class Autosave:
    def __init__(self, **fields):
        self._fields = fields
    
    def __enter__(self):
       self.before = LookupRecordList()
    
    def __exit__(self, A, B, C):
        after = LookupRecordList()
        # Get diff of the keys in before and after dict, then add them to Autosave

Got it, I think this specific implementation would be problematic because of circular imports, but I can see if I can come up with something that achieves the same thing. Of course could maybe avoid the import problem by importing within a class/function scope but could be a bit of a code smell.
edit: never mind, looks like that's fine

[Araneidae]

this specific implementation would be problematic because of circular imports

The simplest fix for that would be to replace

from softioc.device_core import LookupRecordList

with

from softioc import device_core

and invoke as device_core.LookupRecordList: Python handles circular imports by postponing populating a recursively imported module until the imports are all done, so you can import a module recursively so long as you don't look inside it too early! (Maybe you already know all this...)

[jsouter]

I realise I was mistaken about how the device name work, I was under the impression that the device name gets preprended to PVs that were instantiated before builder.SetDeviceName() when builder.LoadDatabase() gets called. Since this is apparently not the case, I will undo the change to strip the device prefix from the autosave key. This will make it much simpler to support the edge case where users change the device name in the middle of the program and define multiple records with the same name (which I have seen in a real softioc in use at Diamond).

Considering the idea of multiple devices within an ioc, would we want to support the ability to backup differnet groups of PVs to multiple files, each with a different save period? I have drafted this with both sync and async methods, and it is a feature provided by C++ autosave.

[AlexanderWells-diamond]

I'm inclined to keep it simple, at least for the first release. If anyone wants the extra feature they can raise an issue for it.

[Araneidae]

Looks good to me. Would like to discuss the Autosave context handler default arguments, but think this is fine.

[AlexanderWells-diamond]

Overall this is good, very close to done. I spotted a couple of potential bugs and there's a few docs changes that I'd like to see before this is finished.

[AlexanderWells-diamond]

Is it expected that the Context Manager's True would override the autosave=False? Or is there some other reason this item is in the saved list?

[jsouter]

That's how it works currently yes, I suppose we could find a way to use some True/False/None logic if we wanted an explicit calling of autosave=False to override the context manager?

[AlexanderWells-diamond]

I see. My view would be that setting autosave=False on the record creation function should override everything else - it's the principle of least surprise to take its value over any other.

However, I can see that may be difficult to actually achieve. I can also see that it seems very unlikely anyone will ever actually try and do that. If there's an easy way to at least print/raise an error in this case then do that, otherwise it is a rather confusing behaviour. @Araneidae, any thoughts?

[jsouter]

Can roll back if required but just pushed something to address this: we call add_pv_to_autosave on all PVs at init now, and class variables are set in the context manager which are used to override the the arguments "autosave" and "autosave_fields" if not specified, so an explicit autosave=False will get applied. It also means we don't have to use LookupRecordList any more.

But I did realise that this newer implementation, and the LookupRecordList implementation would not be thread safe, in the sense that if a separate thread is creating PVs while another thread is creating PVs inside a context manager, we may accidentally track fields in autosave that we don't want to. I'm not sure if anyone would write their IOC that way, but may be something worth trying to guard against.

[AlexanderWells-diamond]

As this data is used in several tests, I'd be tempted to split it out into its own top level dictionary or a fixture, to make it obvious that these values will be references in many tests. Not required, just a nice to have.

[jsouter]

I believe I have managed to make the context managers thread safe by using a singleton style class that inherits from threading.local. I have moved some of the context manager logic into a new _AutosaveContext class, but if this seems a bit too convoluted see the approach in the commit 2da2fbf which applies all the singleton/threading logic to Autosave.
Edit: also noting here that the tests are all passing on my machine, I will make sure to fix the build but have mostly been focusing on the actual code recently

[jsouter]

If we have a non-default list of autosave_fields for a PV inside an Autosave context manager, do we want to combine the lists or give priority to just the specified fields in the PV init? I suppose we still need to discuss if we want to track VAL inside autosave_fields instead of using autosave as its own kwarg.

[AlexanderWells-diamond]

I'd be inclined to merge the list of extra fields. If users want to have an override they can just remove that one PV from the context manager.

Regarding tracking VAL, it's an awkward one due to the assumption that a record name without .VAL is actually referring to the VAL field. If that's also how C++ Autosave works I'm inclined to leave it as it is - having autosave=True is what I expect most users will want, and is shorter than writing autosave_fields=["VAL"].

[AlexanderWells-diamond]

Looks good to me!

The test failures are due to p4p issues with later versions of Python and numpy2.0. They will be fixed when p4p is updated.

[jsouter]

Great! I don't seem to have permissions to merge but happy to have it squashed with a message like "add support for saving and restoring pv fields with autosave"