Alarm Lifecycle
OpSee alarms follow ISA-18.2, the alarm-management standard used in process industry. The user interface uses plain words (snooze instead of shelve), but the states, the priorities and the timing rules below are the standard ones. Knowing them explains why an alarm that already returned to normal is still in the list, or why an alarm does not clear the moment a value drops below its limit.
Definitions, Occurrences and Events
- An alarm definition describes one alarm: the equipment it belongs to, its priority, its guidance texts, and three conditions: set (when it is raised), reset (when it returns to normal) and acknowledge (when it counts as seen). Admins create definitions; see Alarm Administration.
- An occurrence is one instance of the alarm being raised. A definition has at most one open occurrence at a time.
- An event is one thing that happened to an occurrence: set, reset, ack, shelve, unshelve, comment, auto-ack, escalate, notify. Events form the timeline the operator sees in the detail drawer. Five of them — set, reset, ack, shelve and escalate — are routable: they are what notifications react to.
States
| State | Meaning | What the operator sees |
|---|---|---|
| Normal | The set condition is false and there is no open occurrence | Nothing |
| Active unacknowledged | The alarm is raised and nobody has acknowledged it | Row with a colored left border; critical rows blink; counted in the bell |
| Active acknowledged | Still raised, but someone said "seen" | Quiet row, stays in the list until it returns to normal |
| Return-to-normal unacknowledged | The condition cleared before anyone acknowledged it. The occurrence stays open so nobody misses that something happened. | Row with a colored left border; acknowledging closes it |
| Snoozed (shelved) | Hidden from the default list for a fixed duration, at most 7 days, with a reason beyond one hour. Evaluation and history continue. | Visible with Show snoozed; re-appears on expiry if still set |
| Out of service | The whole definition is paused by an admin until a date and time, typically for maintenance. Open occurrences are closed. | Nothing until the window ends |
Hiding an alarm on the operator screen is not a state: it is a per-browser filter that changes nothing on the server.
Priorities
ISA-18.2 recommends at most four priority levels. OpSee uses exactly four:
| Priority | Color | Typical use |
|---|---|---|
| Critical | Red | Immediate action required; safety or major damage |
| High | Orange | Prompt action required |
| Medium | Yellow | Action required during the shift |
| Low | Blue-grey | Information, no immediate action |
The priority is a snapshot on the occurrence: changing the definition's priority affects the next occurrence, not the open one. Every definition also carries three rationalization texts the standard asks for: cause, consequence and operator action.
Condition Kinds
A condition is one {kind, configuration} pair per role. The kind decides how the question is answered; the role decides which question it answers. Only three of the kinds can decide a set — the others exist because "when does it clear" and "when does it count as seen" are different questions from "when does it fire".
| Kind | Roles | What it decides | Configuration |
|---|---|---|---|
| Tag compare | set, reset, ack | One tag against a constant or a range | Tag, operator (<, <=, =, <>, >=, >, between, outside), value or two values, hysteresis, on-delay |
| Tag limit | set, reset | One tag against its own configured limit, read at evaluation time so editing the limit edits the alarm | Tag, limit (USL, UCL, warning high, warning low, LCL, LSL), hysteresis, on-delay |
| Expression | set, reset, ack | A boolean formula over several tags, with time windows (temp > 80 and avg('flow', '5m') < 5) | Condition text, one binding per name (name to tag), on-delay |
| Stale | set | That a tag has produced no value for N seconds — the only kind that looks at the timestamp instead of the value | Tag, max age |
| Inverse | reset | That the set condition is not true any more, with deadband and off-delay. The default reset. | Hysteresis, off-delay |
| Auto | reset, ack | A fixed delay after another transition: acknowledge N s after the return to normal, or reset N s after the set (pulse alarms) | After (on reset / on set), delay |
| Timer | reset | That N seconds have passed since the alarm was raised, whatever the value does | After (s) |
| Manual | reset, ack | That a person did it. The default acknowledge. | — |
| External | — | Not available yet: an alarm raised by another system through the API. The kind exists in the data model but no evaluator supports it, so it is refused rather than stored as an alarm that could never fire. | — |
When a definition has no reset row it behaves as inverse, and with no acknowledge row as manual. So the smallest possible definition is a single set condition — which is exactly what a fast alarm creates.
Two properties hold for every kind:
- A condition that cannot be decided holds the current state. A missing sample, a null or bad-quality value, an empty window, a tag of the wrong type or a failed evaluation never raises an alarm and never clears one; the delay timers freeze and the confirmed state is kept until real values arrive. Only stale treats absence as an answer.
- Delays and hysteresis belong to the condition, not to the alarm. The set condition carries the on-delay, the reset condition the off-delay, and both can carry hysteresis; see the next section.
The expression kind is the one that reads more than one tag: every name in the formula is bound to a tag, and those bound tags are tags of the definition — they are watched by the evaluator and guarded by the same equipment scope as the definition itself. Its window functions (avg, sum, wmin, wmax, count, delta over 30s / 5m / 1h / 2d, plus last) are computed once per cycle for every expression at the same time, so a formula over a five-minute average costs the evaluator one aggregate query, not one per alarm. The syntax and its rules are in Alarm Administration.
On-Delay, Off-Delay and Hysteresis
Three settings keep a noisy value from producing a chattering alarm.
| Setting | Where | Effect |
|---|---|---|
| On-delay | Set condition | The set condition must stay true for this many seconds before the alarm is raised. Suppresses momentary spikes. |
| Off-delay | Reset condition | The reset condition must stay true for this many seconds before the alarm returns to normal. Stops flapping around the limit. |
| Hysteresis | Set (compare, limit) and reset (inverse) conditions | Once the alarm is set, the threshold is relaxed by the deadband before the alarm can reset |
Worked example
Definition: raise when Temperature > 80, hysteresis 2, on-delay 10 s, off-delay 5 s (the system default).
| Time | Value | What the evaluator does |
|---|---|---|
| 00:00 | 79.5 | Set condition false. Normal. |
| 00:05 | 80.4 | Set condition true. On-delay timer starts. |
| 00:12 | 79.8 | Set condition false again. Timer is discarded. No alarm: the spike lasted 7 s, less than the 10 s on-delay. |
| 00:20 | 81.0 | Set condition true. Timer starts. |
| 00:30 | 82.3 | True for 10 s. Alarm raised, trigger value 82.3. |
| 00:45 | 79.0 | Because of hysteresis 2 the alarm only resets at <= 78. 79.0 is still inside the deadband: the alarm stays active. |
| 01:00 | 77.5 | Reset condition true (<= 78). Off-delay timer starts. |
| 01:05 | 77.9 | True for 5 s. Return to normal. The occurrence closes if it was acknowledged, otherwise it stays open as return-to-normal unacknowledged. |
Without hysteresis the alarm would have reset at 00:45 and been raised again on the next reading above 80.
When a condition does not set its own delay, the values of ALARM_DEFAULT_ON_DELAY_S (default 0) and ALARM_DEFAULT_OFF_DELAY_S (default 5) apply; see System Parameters.
When the condition really does flap
Delays and hysteresis handle a noisy value. They cannot help a condition that genuinely comes and goes over minutes — a failing sensor, an intermittent link, a process cycling around its limit. Because an occurrence stays open until it is acknowledged, each return of the condition re-arms the same occurrence instead of raising a new one, and the alarm can cycle indefinitely.
Two things make that bearable:
- The alarm counts its re-arms. The detail drawer shows Re-armed N× and when it last happened, the operator grid carries the same number in its
×Nchatter badge, and the admin KPIs count re-arms as annunciations — in the rate, in the flood percentage, and as their own column in the chattering top ten. All of them are the number to look at when Raises in 24 h says0, because a re-arm creates no new occurrence for that count to see. - Re-arms are not notified. Subscribers and alarm actions hear about the set that raised the occurrence, its first return to normal and the clear that closes it, and nothing for each flap in between, so a chattering alarm cannot flood a phone. Escalation fires once per occurrence.
- Every set/reset pair is written to the occurrence's timeline by default (
ALARM_CHATTER_EVENT_LIMIT=0, no limit), so the history shows every cycle. An administrator can set a limit to stop recording re-arms past that many pairs; the state changes and the re-arm counter are unaffected either way.
The alarm itself is never silenced: it changes state exactly as it would otherwise, stays in the alarm list, and the re-arm counter keeps rising. Acknowledging the occurrence is what ends the cycle — a re-arm only happens while it is unacknowledged.
Stale Watchdog
A stale set condition raises the alarm when a tag has produced no value for max_age_s seconds. It looks at the timestamp of the last sample, not its value, so it is the only condition that reacts to the absence of data: a stopped collector, a broken sensor, a dead network link. The fast-alarm preset No new value for N min creates one with a default of 5 minutes and medium priority.
Latched and Auto-Acknowledge
- A latched alarm (the default) stays open until a person acknowledges it, even after it returned to normal. This is the ISA-18.2 behavior: the operator must see that something happened while they were away.
- An unlatched alarm is acknowledged automatically the moment it returns to normal, so the occurrence closes without anyone clicking. Use it for informational alarms that need no follow-up.
- The acknowledge condition can also be Auto (acknowledge N seconds after reset, or after set) or a Tag compare (for example a PLC reset button), instead of the default Manual.
Null and Bad-Quality Values
A missing sample or a null (bad-quality) value never sets an alarm and never resets one. The evaluator treats it as indeterminate: delay timers are frozen and the current state is held until a real value arrives. Only the stale watchdog reacts to missing data. When a stale alarm is raised its trigger value is empty.
An expression condition extends the same rule to everything that can make a multi-tag formula unanswerable: one bound tag without a value, a null, a text or timestamp tag, a window with no rows in it, or an evaluation error such as a division by zero. Any of them makes the whole expression indeterminate for that cycle, so it holds the state instead of guessing. A multi-tag alarm therefore fails silent rather than fails loud — which is why a bound tag that goes quiet deserves a stale alarm of its own.
The Evaluator
The alarm_evaluator worker runs one cycle every second (its poll interval in System Parameters):
- Load the enabled definitions that are not out of service (refreshed every 60 s and immediately when an admin saves one).
- Read the tag values that arrived since the previous cycle, for the watched tags only.
- Run each definition's state machine: compare, apply hysteresis, on-delay and off-delay, and check timers, auto and stale conditions that need no incoming value.
- Write the transitions (raise, clear, acknowledge) in one transaction, add the events, un-snooze expired occurrences, and notify the browsers so the Alarms screen and the bell update live.
Because it reads values from the database it sees every source in the same way: MQTT, OPC-UA, Modbus, manual writes and calculated tags. An alarm on a calculated OEE or rate needs no extra configuration. Open occurrences live in the database, so a restart of the worker keeps them; only delay timers that were running restart.
Latency from a value arriving to the alarm being raised is one cycle plus the on-delay, which is adequate for operator alarms.
Notifications and Escalation
Delivery is decoupled from evaluation. The evaluator writes events; a second worker, the alarm_notifier, turns the routable ones (set, reset, ack, shelve, escalate) into messages and delivers them through channels, without ever slowing the evaluator down. Every message goes through an outbox row with its own small lifecycle:
| State | Meaning | Where it shows |
|---|---|---|
| Pending (queued) | Matched and waiting for the notifier, or waiting for its retry time | Detail drawer, Notifications sent, chip Queued |
| Sending | Claimed by one notifier instance, delivery in progress (30 s ceiling per send) | Chip Sending |
| Sent | Delivered; the timeline gets a notification event | Chip Sent, timeline |
| Failed | Every attempt failed (NOTIFY_RETRY_MAX_ATTEMPTS, backoff NOTIFY_RETRY_BACKOFF_S × attempt); the reason is stored on the row, on the channel's Last error and in the timeline | Chip Failed with the attempt count |
| Summarised | Held back by flood control (NOTIFY_MAX_PER_CHANNEL_PER_MINUTE) and folded into one summary message per NOTIFY_FLOOD_SUMMARY_WINDOW_S | Not listed individually; the summary is one message on the channel |
An escalation is a timer rather than a state. When an occurrence is raised, every enabled Escalated action on its definition arms a timer for its delay. When the timer elapses the notifier asks the database whether the occurrence is still unacknowledged (active or return-to-normal unacknowledged); only then is the escalation message queued and the timeline gets an escalate event. Acknowledging, snoozing or closing the occurrence cancels its pending timers, so an alarm that was handled in time never escalates. A re-raise after a return to normal re-arms them.
Notifications for set and escalate carry a signed acknowledge link that is valid for 24 hours and works only while the occurrence is unacknowledged; see Acknowledging from your phone. Channels, routing rules and actions are described in Alarm Notifications.
Retention
Occurrences are kept for RETENTION_ALARM_DAYS (default 365; 0 keeps them forever). The retention worker drops whole 7-day chunks of occurrences, so the effective horizon can exceed the setting by up to one chunk. The event timeline rows (set, ack, comment, and so on) live in a plain table, so the same parameter prunes them by exact age instead of by chunk — with one carve-out: the timeline of an occurrence that is still open is kept regardless of age, because it is what the operator reads in the detail drawer and a long-standing unacknowledged alarm must not lose its history. Disabling or deleting a definition closes its occurrences but does not delete the history. Settled notification outbox rows follow a separate RETENTION_NOTIFICATION_DAYS (default 90), deleted by the notifier itself; the notification events in the timeline stay.
Next steps
- Alarms Screen: how operators acknowledge, snooze and comment
- Alarm Administration: how to build the set, reset and acknowledge conditions
Related
- Data Flow: where the tag values the evaluator reads come from
- System Parameters: default delays, retention and the alarm workers
- Glossary