[stormshield] Type the appliance health counters#20229
Draft
arenard wants to merge 3 commits into
Draft
Conversation
…metadata allow-list Derive `event.outcome` on the auth, server and xvpn families from the vendor `error` return code (`0` maps to success, other codes to failure). The official field documentation lists exactly these three families for `error`; on vpn logs the same key is a severity level and is left untouched. The vendor documentation pairs `0`-first numeric examples with `Success`-first display labels without stating the convention outright, so the mapping follows the strong implication of the field tables. Map `domain` to `user.domain` and `usergroup` to `user.group.name`, splitting the comma-separated multi-valued form that SNS 5.x emits. On web logs, map `op` to `http.request.method` and `result` to `http.response.status_code`; on other families these keys carry protocol-specific codes (FTP, Modbus) and are not promoted. Map the module name in `service` to `process.name` on the system, sandboxing and routing families only — on pvm logs the same key names the vulnerable product detected on a scanned host. Set `observer.product`. All mappings are copies; the vendor keys remain available under `stormshield.metadata`. Extend the `stormshield.metadata` allow-list with the documented vendor keys that previously leaked as undeclared `stormshield.*` fields: host and IP reputation (`srchostrep`, `dsthostrep`, `srciprep`, `dstiprep`), alarm packet captures (`pktlen`, `pktdump`, `pktdumplen`), Ethernet protocol (`etherproto`), SSO agent (`agentid`), IKE peer name (`peername`), and the industrial-protocol plugin fields (`unitid`/`unit_id`, `cipservicecode`, `cipclassid`, `requestmode`, `responsemode`, `UI`, `error_class`, `error_code`, `format`). The NAT reading of the pipeline was verified against the official field tables while preparing this change: `origdst`/`origdstport` are documented as the destination before translation and map to `destination.ip`/`destination.port`; `dst` is the translated destination and maps to `destination.nat.ip` when it differs. A connection sample with the documented example values now locks this in the pipeline tests, alongside new samples for the alarm reputation and packet-capture fields, a Modbus plugin line (whose `result` stays under metadata and does not produce `http.*`), a server line, a system line, and multi- and single-valued `usergroup` forms.
Declare typed stormshield.* fields for the documented vendor keys that carry detection value instead of routing them to the flattened metadata object: the alarm context (class, classification, repeat, sensible, target, icmptype, icmpcode) and packet capture (pktlen, pktdumplen, pktdump), the IP and host reputation keys (srciprep, dstiprep, srchostrep, dsthostrep), the IKE negotiation keys of IPsec VPN logs (phase, side, ikev, spi_in, spi_out, cookie_i, cookie_r, localnet, remotenet, remoteid), the authentication keys (method, totp, confid, agentid), the administration session identifier (sessionid), the filter slot (slotlevel, slotname), the Ethernet protocol (etherproto), the industrial and REST protocol identifiers (unitid, cipservicecode, cipclassid, requestmode, responsemode, format, error_class, error_code), the web filtering keys (cat_site, urlruleid, contentpolicy), the antivirus, antispam and sandboxing verdicts (virus, spamlevel, filename, filetype, sandboxing, sandboxinglevel) and the application identification keys (clientappid, serverappid, groupid). Numeric keys are converted to long so they stay aggregatable, the comma-separated IP reputation keys are split into arrays and the curly braces of the vendor cat_site format are removed. spamlevel stays keyword because the documented values include X and ?, and contentpolicy keeps its zero-padded form. The field descriptions follow the wording of the official audit log documentation, cross-checked against the v4 and v5 field tables; the keys that v5 no longer documents are marked as such. The error key stays under stormshield.metadata where the outcome derivation reads it. Most of the moved keys reached the flattened object in the 1.6.0 and 1.7.0 releases, so relocating them now, before those releases see wide adoption, keeps the migration cost minimal. Add pipeline test samples derived from the official documentation examples for ICMP alarms, for the NTP and S7 protocol inspections, and for the antivirus, antispam and sandboxing verdicts on an SMTP flow.
Type the counters of the periodic statistics logs as aggregatable numbers instead of flattened strings. The scalar counters of the filter statistics log (accepted and blocked totals, memory and state gauges) move to long fields under stormshield.filterstat, keeping the names the vendor writes. The CPU counters of the monitor log (time allocated to user processes, kernel time and system disruptions, as the official documentation describes them) move to stormshield.monitor.cpu, and the security level of the monitor log becomes the long field stormshield.security. Counters whose value does not parse as a number fall back to the flattened metadata object unchanged, and the composite in/out pairs stay under metadata where the existing byte extraction reads them. These fields feed appliance health dashboards and alerting on CPU time, memory gauges and blocked packet rates, which the flattened object could not serve. The per-method authentication counters of the authstat log stay under metadata: the official field tables do not document them, and the per-interface structures of the monitor log stay as well, pending a document layout that supports per-interface aggregation.
arenard
force-pushed
the
stormshield-stats-typing
branch
from
July 20, 2026 11:05
a5cdfe0 to
0669795
Compare
Contributor
✅ Elastic Docs Style Checker (Vale)No issues found on modified lines! The Vale linter checks documentation changes against the Elastic Docs style guide. To use Vale locally or report issues, refer to Elastic style guide for Vale. |
Contributor
|
Changelog link mismatch — expected
Tip If expected, add the |
💔 Build Failed
Failed CI StepsHistory |
Contributor
🚀 Benchmarks reportTo see the full report comment with |
andrewkroh
added
documentation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Proposed commit message
Type the counters of the periodic statistics logs as aggregatable numbers instead of flattened strings. The scalar counters of the filter statistics log (accepted and blocked totals, memory and state gauges) move to long fields under
stormshield.filterstat, keeping the names the vendor writes. The CPU counters of the monitor log (time allocated to user processes, kernel time and system disruptions, as the official documentation describes them) move tostormshield.monitor.cpu, and the security level of the monitor log becomes the long fieldstormshield.security. Counters whose value does not parse as a number fall back to the flattened metadata object unchanged, and the composite in/out pairs stay under metadata where the existing byte extraction reads them.These fields feed appliance health dashboards and alerting on CPU time, memory gauges and blocked packet rates, which the flattened object could not serve. The per-method authentication counters of the authstat log stay under metadata: the official field tables do not document them, and the per-interface structures of the monitor log stay as well, pending a document layout that supports per-interface aggregation.
Design notes
stormshield.filterstatis declared as an object of longs: the vendor counter set varies with the appliance features in use, and every member is a numeric gauge or total. The numeric guard routes any non-numeric value tostormshield.metadatainstead of failing the document.CPUfield entry (CPU=1,0,2: user processes, kernel, system disruptions) and were already used by the monitor sub-pipeline.stormshield.system(a percentage, keyword since the package's first release) is left untouched to avoid a mapping type change on a released field; it is a candidate for the next breaking-change release.Checklist
changelog.ymlfile.Author's Checklist
elastic-package check,test pipelineandtest system(udp, tcp, tls) pass against stack 9.4.3How to test this PR locally
Related issues