Draft: Add security violations filter processor, and plug it in to the default OTLP logs pipeline when TcplogReceiver is configured#1576
Open
kamalchaturvedi wants to merge 2 commits intonginx:mainfrom
Conversation
…st message as a one-time gate — on the very first log record, it checks: - The body is a string (not int, bytes, etc.). - The body has exactly 27 pipe-separated fields matching the secops_dashboard-log profile format. If either check fails, the gate closes permanently — all subsequent messages are dropped with zero overhead (early return before any iteration) until the OTel collector is restarted. If it passes, the gate opens permanently and all future string-bodied records flow through.
…ne, replacing the: - logsgzipprocessor: This processor was introduced for NGINX One for a deprecated project, which is not in scope anymore. - securityviolationsprocessor: This processor was converting the sylog security violation into a JSON body log record format, and then forwarding it. This deterministed JSON log body parsing is an overhead for the agent to process for every single log record.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Proposed changes
COMMIT 1:
Add a new security violations filter processor that validates the first message as a one-time gate — on the very first log record, it checks:
- The body is a string (not int, bytes, etc.).
- The body has exactly 27 pipe-separated fields matching the secops-dashboard-log profile format.
COMMIT 2:
Plug security violation filter processor into the default logs pipeline, replacing the:
- logsgzipprocessor: This processor was introduced for NGINX One for a deprecated project, which is not in scope anymore.
- securityviolationsprocessor: This processor was converting the sylog security violation into a JSON body log record format, and then forwarding it. This deterministix JSON log body conversion is an overhead for the agent to process for every single log record.
Checklist
Before creating a PR, run through this checklist and mark each as complete.
CONTRIBUTINGdocumentmake install-toolsand have attached any dependency changes to this pull requestREADME.md)