Conditional latching of alarms

[georgweiss]

We're seeing issues with IOCs connecting to PLCs and where the (TCP/IP) connection times out and goes away, and then reconnects a brief moment later. Disconnection triggers in many cases an avalanche of INVALID alarms, obscuring other (more "important") alarms.

To help operators with the visual clutter, it has been proposed to support conditions on the latching property: e.g. set latching = false if alarm state is INVALID.

Intention of this discussion is to collect feed-back and to identify potential pitfalls and side effects.

[kasemir]

"Maintenance Mode" suppresses annunciation of invalid/unknown alarms and auto-acknowledges them.
It's meant for maintenance periods where equipment is offline or restarted and those invalid/unknown alarms are expected.

Operationally, however, it will be difficult to distinguish between PVs that are invalid/unknown because of known reasons that should be ignored or equipment being on fire, where you might want to follow up.

reconnects a brief moment later.

Maybe just configure the alarm delay long enough to bridge the known hiccups for the PLC PVs?
Ideally, of course, fix the PLC support.
What types of PLCs are these? Using what protocol? modbus, OPC/CA, Siemens,...?

[georgweiss]

I've been told there is a heartbeat.

[georgweiss]

Configuring a delay would affect MAJOR/MINOR as well, and that does not seem to be acceptable.

[kasemir]

Still, I understand that shooting the messenger is fun, but this looks like a known problem at the source, needs to be fixed there.

[georgweiss]

a known problem at the source

Indeed.

[kasemir]

If it can't be fixed in the S7PLC driver, handle it in the alarm trigger PV, which might also address

.. an avalanche of INVALID alarms

Given some PLC-controlled subsystem like a water cooling system, it's usually NOT a good idea to add every temperature sensor, flow sensor, valve position PV into the alarm server. Instead, summarize the info into a "Cooling system is OK" type of PV and add just that to the alarm server. Technically, that summarization is implemented via a bunch of calc records on the IOC.
The underlying idea here: If you knew exactly what to do about some elevated temperature reading or low flow PV value, then you should have automated the response in the first place, there's nothing left to do for the alarm system. The alarm comes in when you don't know what to do. All you know is that one or more temperatures, flows, valve positions are out of their normal range, the cooling system is NOT OK, and needs a knowledgable operator to investigate. You want one "Cooling system is NOT OK" alarm from that, with a link to a screen that shows all the temperature, flow, valve info for the system, proving the operator with a good starting point to investigate. Not 40 alarms, one for each PV on that screen.
In the summarization, you could ignore invalid/unknown to work around the disconnect issue, at least temporarily.