feat(logs pipelines): surface the collector logs in the preview - #12535
Draft
tushar-signoz wants to merge 2 commits into
Draft
feat(logs pipelines): surface the collector logs in the preview#12535tushar-signoz wants to merge 2 commits into
tushar-signoz wants to merge 2 commits into
Conversation
tushar-signoz
force-pushed
the
tvats-fix-json-pipelines-preview
branch
from
August 12, 2026 15:35
96ac129 to
f235791
Compare
tushar-signoz
force-pushed
the
tvats-pipelines-collector-logs
branch
from
August 12, 2026 15:36
ec5b281 to
44173e9
Compare
The collector gets a normalize pipeline prepended ahead of user pipelines when use_json_body is on, delivered over opamp, so a pipeline authored against body.<field> behaved differently in preview than in production. Preview evaluates the flag for the caller's org and prepends the same pipeline.
tushar-signoz
force-pushed
the
tvats-fix-json-pipelines-preview
branch
from
August 12, 2026 20:40
f235791 to
5b5ef4d
Compare
- return what the collector logged while simulating, so an operator that could not parse a log explains itself instead of silently changing nothing - drop the stack trace lines the collector emits after each error, which arrive as one entry per frame
tushar-signoz
force-pushed
the
tvats-pipelines-collector-logs
branch
from
August 12, 2026 20:41
44173e9 to
0d50887
Compare
pull Bot
pushed a commit
to Mu-L/signoz
that referenced
this pull request
Aug 13, 2026
…igNoz#12520) #### Description - The collector gets a `normalize` pipeline prepended ahead of user pipelines when `use_json_body` is on — injected in `RecommendAgentConfig` and delivered over opamp — which parses the log body into JSON. Preview simulated only the user's pipelines, so a pipeline authored against `body.<field>` behaved differently in preview than in production, and one written against `body` looked fine in preview while doing nothing on real logs. - Preview now evaluates the flag for the caller's org and prepends the same pipeline, so what it shows is what the collector does. #### Issues closed by this PR Fixes SigNoz/engineering-pod#5897 #### Additional Information Verified end to end against a local stack — devenv ClickHouse, a collector with `body_json_enabled` connected over opamp, `use_json_body` on — by driving the two calls the preview screen makes: sample logs, then preview with those logs. `parse_from: body.message` extracts attributes; `parse_from: body` extracts nothing, matching what the collector does with a normalized body. Log bodies render as stored rather than unwrapped, so what you see is what the pipeline operates on. Needs SigNoz#12534 to pick sample logs by body — without it the v3 query behind the sample-log list errors for these orgs. SigNoz#12535 stacks on this to surface the collector's own explanation when an operator cannot parse a log.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
collectorLogs— what the collector logged while simulating — but the UI dropped the field. They are now shown under the previewed output, so a pipeline that silently does nothing explains itself.Additional Information
The case this exists for: with
use_json_bodyon, the collector runs anormalizepipeline before user operators, so the body is a map by then. A parser pointed atbodyinstead ofbody.messagecannot work — it extracts nothing and, until now, said nothing. Driven against a local collector, preview withparse_from: bodyreturns empty attributes plus:Stacked on #12520 — retarget to
mainonce that merges.