Skip to content

feat: rewrite telemetery and configs - #18

Merged
oneslash merged 7 commits into
mainfrom
feat/rewrite-telemetry
Jul 11, 2026
Merged

feat: rewrite telemetery and configs#18
oneslash merged 7 commits into
mainfrom
feat/rewrite-telemetry

Conversation

@oneslash

@oneslash oneslash commented Jul 11, 2026

Copy link
Copy Markdown
Collaborator

Summary by CodeRabbit

  • New Features

    • Added telemetry.mode to scenarios (required default) and telemetry: { mode: off } for output-only comparisons.
    • Added --telemetry-mode {required|off} for limier run and limier ci github.
    • Reports and rendered output now include telemetry mode/status/sensors (including in completion logs).
  • Bug Fixes

    • When required telemetry can’t start or complete, runs are marked inconclusive with rerun guidance; output-only mode always requires human review.
  • Documentation

    • Updated Linux telemetry prerequisites, CI/runner guidance, and scenario/CLI references to use telemetry.mode semantics.
  • Tests

    • Added unit tests and a Linux eBPF integration test covering telemetry behavior.

@coderabbitai

coderabbitai Bot commented Jul 11, 2026

Copy link
Copy Markdown

Review Change Stack

📝 Walkthrough

Walkthrough

Telemetry configuration replaces host-signal capture with required or off modes. The mode flows from scenario files and CLI flags through collection, diagnostics, reports, rendered output, tests, and operational documentation.

Changes

Telemetry mode migration

Layer / File(s) Summary
Scenario telemetry contract
internal/scenario/..., internal/preset/assets/scenarios/*, scenarios/npm.yml, docs/reference/scenario-file.md
Scenario telemetry defaults to required, supports off, validates modes, rejects unknown fields, and replaces the prior evidence configuration.
CLI and runtime execution
cmd/run.go, cmd/ci.go, internal/limier/run.go, internal/limier/run_test.go
CLI overrides are resolved and passed into execution; required mode activates collection, while disabled or failed telemetry updates diagnostics, coverage, recommendations, and exit status.
Collector lifecycle and validation
internal/collector/*, internal/analysis/analysis.go, internal/env/docker/manager_ebpf_integration_test.go
Process-exec events use a shared constant; bpftrace readiness, output draining, shutdown synchronization, diagnostics, and Linux eBPF integration coverage are updated.
Report and rendered output
internal/report/*, internal/render/*
Reports and markdown summaries include telemetry mode, status, and sensors, with disabled telemetry explicitly requiring human review.
Guidance and diagrams
README.md, docs/guide/*, docs/launch-readiness.md, .agents/skills/limier-cli/SKILL.md, docs/diagrams/*
Documentation and operation-flow diagrams describe Linux kernel telemetry prerequisites, telemetry.mode, fail-closed behavior, and output-only review semantics.

Estimated code review effort: 4 (Complex) | ~45 minutes

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title is clearly about the telemetry/configuration rewrite and matches the main scope of the changeset.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/rewrite-telemetry

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 4

🧹 Nitpick comments (3)
internal/report/report.go (2)

1-1: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Duplicated rule-hit rendering loop across report.go and render.go.

Both files copy-pasted the identical rule-hit formatting loop when adding the telemetry-disabled branch. Extracting a small appendRuleHitLines helper in each file eliminates the duplication.

  • internal/report/report.go#L354-372: extract a local appendRuleHitLines(&lines, runReport.RuleHits) helper and call it in both the disabled-telemetry branch (lines 356-362) and the default branch (lines 366-372).
  • internal/render/render.go#L109-117: extract the same local helper and call it in both the disabled-telemetry branch (lines 111-117) and the default branch (lines 121-127).
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@internal/report/report.go` at line 1, Extract the duplicated rule-hit
formatting loops into an appendRuleHitLines helper in both report.go and
render.go. Update the telemetry-disabled and default branches in the report and
render flows to call the respective helper with the output lines and RuleHits,
preserving the existing formatting and branch behavior.

354-372: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Extract duplicated rule-hit rendering into a helper.

The rule-hit formatting loop at lines 356-362 is identical to lines 366-372. The same duplication appears in internal/render/render.go (lines 111-117 vs 121-127). Extracting a small helper eliminates the copy-paste in both files.

♻️ Proposed helper extraction
 func BuildSummary(runReport Report) string {
 	var lines []string
 	// ... existing code ...
 	lines = append(lines, "", "## Why This Verdict", "")
 	if runReport.Diagnostic != nil {
 		lines = append(lines, "- "+runReport.Diagnostic.Summary)
 	} else if runReport.Telemetry.Status == TelemetryStatusDisabled {
 		lines = append(lines, "- Kernel-level telemetry was disabled, so this comparison requires human review.")
-		for _, hit := range runReport.RuleHits {
-			line := fmt.Sprintf("- %s matched `%s`", hit.Category, hit.RuleID)
-			if strings.TrimSpace(hit.Reason) != "" {
-				line += ": " + hit.Reason
-			}
-			lines = append(lines, line)
-		}
+		appendRuleHitLines(&lines, runReport.RuleHits)
 	} else if len(runReport.RuleHits) == 0 {
 		lines = append(lines, "- No rules matched. The recommendation comes from the raw diff outcome.")
 	} else {
-		for _, hit := range runReport.RuleHits {
-			line := fmt.Sprintf("- %s matched `%s`", hit.Category, hit.RuleID)
-			if strings.TrimSpace(hit.Reason) != "" {
-				line += ": " + hit.Reason
-			}
-			lines = append(lines, line)
-		}
+		appendRuleHitLines(&lines, runReport.RuleHits)
 	}
 	// ...
 }
+
+func appendRuleHitLines(lines *[]string, hits []RuleHit) {
+	for _, hit := range hits {
+		line := fmt.Sprintf("- %s matched `%s`", hit.Category, hit.RuleID)
+		if strings.TrimSpace(hit.Reason) != "" {
+			line += ": " + hit.Reason
+		}
+		*lines = append(*lines, line)
+	}
+}
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@internal/report/report.go` around lines 354 - 372, The rule-hit rendering
loop is duplicated across the telemetry-disabled and normal branches in the
report generation flow, and similarly in render generation. Extract the shared
hit-formatting logic into a small helper in each relevant package, then call it
from both branches while preserving the existing category, rule ID, and optional
reason formatting.
internal/render/render.go (1)

109-117: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Rule-hit rendering loop duplicated from lines 121-127.

This is the same copy-paste pattern flagged in internal/report/report.go. Extracting a shared helper (or a local appendRuleHitLines) would eliminate the duplication in both files.

♻️ Proposed local helper for render.go
 	} else if runReport.Telemetry.Status == report.TelemetryStatusDisabled {
 		lines = append(lines, "- Kernel-level telemetry was disabled, so this comparison requires human review.")
-		for _, hit := range runReport.RuleHits {
-			line := fmt.Sprintf("- %s matched `%s`", hit.Category, hit.RuleID)
-			if strings.TrimSpace(hit.Reason) != "" {
-				line += ": " + hit.Reason
-			}
-			lines = append(lines, line)
-		}
+		appendRuleHitLines(&lines, runReport.RuleHits)
 	} else if len(runReport.RuleHits) == 0 {
 		lines = append(lines, "- No rules matched. This output mirrors the report-level verdict and recommendation.")
 	} else {
-		for _, hit := range runReport.RuleHits {
-			line := fmt.Sprintf("- %s matched `%s`", hit.Category, hit.RuleID)
-			if strings.TrimSpace(hit.Reason) != "" {
-				line += ": " + hit.Reason
-			}
-			lines = append(lines, line)
-		}
+		appendRuleHitLines(&lines, runReport.RuleHits)
 	}
 
+func appendRuleHitLines(lines *[]string, hits []report.RuleHit) {
+	for _, hit := range hits {
+		line := fmt.Sprintf("- %s matched `%s`", hit.Category, hit.RuleID)
+		if strings.TrimSpace(hit.Reason) != "" {
+			line += ": " + hit.Reason
+		}
+		*lines = append(*lines, line)
+	}
+}
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@internal/render/render.go` around lines 109 - 117, Extract the duplicated
RuleHits formatting loop from the telemetry-disabled branch and the
corresponding nearby branch into a shared helper or local appendRuleHitLines
function, then call it from both locations. Preserve the existing output format,
including the optional trimmed Reason suffix, and apply the same deduplication
pattern in internal/report/report.go.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@docs/diagrams/limier-operation-flow.scene-spec.json`:
- Line 317: Make the telemetry requirement conditional in both diagram
artifacts: update the callout label in
docs/diagrams/limier-operation-flow.scene-spec.json at lines 317-317 to say
process-execution telemetry is included when enabled, and synchronize the
rendered text and originalText in docs/diagrams/limier-operation-flow.excalidraw
at lines 3083-3089 with the same wording.

In `@docs/diagrams/limier-operation-flow.verification.json`:
- Line 68: Restore the minimumEditableElements threshold to its previous value
of 18 for this diagram. Do not lower the verification gate to 2 unless the
intended requirement is documented and covered by tests.

In `@docs/guide/getting-started.md`:
- Around line 12-17: Update the prerequisites and kernel telemetry warning in
the getting-started guide to require Linux with bpftrace, cgroup v2, and eBPF
support. Extend the fallback condition to recommend telemetry.mode: off when any
of these prerequisites are unavailable, while preserving the existing
output-only and good_to_go behavior.

In `@docs/launch-readiness.md`:
- Line 41: Update the reviewer journey near the hosted/self-hosted telemetry
guidance to remove stdout/stderr-only and bpftrace comparisons. Describe the
behavior using the documented telemetry.mode values: required must enforce
kernel telemetry availability with fail-closed behavior, while off must
represent output-only review without kernel telemetry; keep the existing
launch-readiness context intact.

---

Nitpick comments:
In `@internal/render/render.go`:
- Around line 109-117: Extract the duplicated RuleHits formatting loop from the
telemetry-disabled branch and the corresponding nearby branch into a shared
helper or local appendRuleHitLines function, then call it from both locations.
Preserve the existing output format, including the optional trimmed Reason
suffix, and apply the same deduplication pattern in internal/report/report.go.

In `@internal/report/report.go`:
- Line 1: Extract the duplicated rule-hit formatting loops into an
appendRuleHitLines helper in both report.go and render.go. Update the
telemetry-disabled and default branches in the report and render flows to call
the respective helper with the output lines and RuleHits, preserving the
existing formatting and branch behavior.
- Around line 354-372: The rule-hit rendering loop is duplicated across the
telemetry-disabled and normal branches in the report generation flow, and
similarly in render generation. Extract the shared hit-formatting logic into a
small helper in each relevant package, then call it from both branches while
preserving the existing category, rule ID, and optional reason formatting.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 578dab85-aaa4-4c4c-ba2c-3333613172f2

📥 Commits

Reviewing files that changed from the base of the PR and between be8b580 and 6e11673.

⛔ Files ignored due to path filters (2)
  • docs/diagrams/limier-operation-flow.preview.png is excluded by !**/*.png
  • docs/diagrams/limier-operation-flow.svg is excluded by !**/*.svg
📒 Files selected for processing (35)
  • .agents/skills/limier-cli/SKILL.md
  • README.md
  • cmd/ci.go
  • cmd/run.go
  • docs/diagrams/limier-operation-flow.excalidraw
  • docs/diagrams/limier-operation-flow.scene-spec.json
  • docs/diagrams/limier-operation-flow.verification.json
  • docs/guide/ci-and-deploy.md
  • docs/guide/getting-started.md
  • docs/guide/review-your-own-project.md
  • docs/guide/understand-results.md
  • docs/launch-readiness.md
  • docs/reference/cli.md
  • docs/reference/scenario-file.md
  • internal/analysis/analysis.go
  • internal/collector/collector.go
  • internal/collector/factory_linux.go
  • internal/collector/factory_nonlinux.go
  • internal/limier/run.go
  • internal/limier/run_test.go
  • internal/preset/assets/scenarios/cargo-ci.yml
  • internal/preset/assets/scenarios/npm-ci.yml
  • internal/preset/assets/scenarios/pip-ci.yml
  • internal/render/render.go
  • internal/render/render_test.go
  • internal/render/testdata/build-summary.golden.md
  • internal/render/testdata/github-comment.golden.md
  • internal/render/testdata/gitlab-note.golden.md
  • internal/render/testdata/inspect-conclusive.golden.md
  • internal/render/testdata/inspect-inconclusive.golden.md
  • internal/report/report.go
  • internal/report/report_test.go
  • internal/scenario/scenario.go
  • internal/scenario/scenario_test.go
  • scenarios/npm.yml

Comment thread docs/diagrams/limier-operation-flow.scene-spec.json Outdated
Comment thread docs/diagrams/limier-operation-flow.verification.json Outdated
Comment thread docs/guide/getting-started.md Outdated
Comment thread docs/launch-readiness.md

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
internal/collector/factory_linux.go (1)

167-220: 🗄️ Data Integrity & Integration | 🔵 Trivial | ⚡ Quick win

Fixed 250ms drain grace is a blind heuristic, not a deterministic completion signal.

waitForBpftraceDrain always sleeps a flat bpftraceDrainGrace (250ms) before signaling stop(), regardless of whether bpftrace has actually finished flushing. This is a tradeoff: too short risks dropping trailing events for slow/loaded hosts (silently degrading telemetry evidence used for "required" mode gating); applied on every step, it also adds up to real latency for CI pipelines with many steps.

Consider whether a deterministic signal (e.g., an explicit "step done" marker printed by the script once the traced cgroup's steady-state is reached) is feasible instead of/in addition to the fixed sleep, or document why 250ms was chosen as sufficient across environments.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@internal/collector/factory_linux.go` around lines 167 - 220, Replace the
fixed-delay waitForBpftraceDrain heuristic used by bpftraceStepCapture.Finish
with a deterministic completion signal from the bpftrace script, such as an
explicit step-done marker after the traced cgroup reaches steady state, and wait
for that signal before calling stop. Ensure the signal is handled with context
cancellation and preserve the existing error propagation and event-draining
behavior.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@internal/collector/factory_linux.go`:
- Around line 167-220: Replace the fixed-delay waitForBpftraceDrain heuristic
used by bpftraceStepCapture.Finish with a deterministic completion signal from
the bpftrace script, such as an explicit step-done marker after the traced
cgroup reaches steady state, and wait for that signal before calling stop.
Ensure the signal is handled with context cancellation and preserve the existing
error propagation and event-draining behavior.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: cefe70b5-922a-4216-82dc-95685628e7c7

📥 Commits

Reviewing files that changed from the base of the PR and between e6c116c and 58de56e.

📒 Files selected for processing (1)
  • internal/collector/factory_linux.go

@oneslash
oneslash merged commit bfb89a9 into main Jul 11, 2026
7 checks passed
@oneslash
oneslash deleted the feat/rewrite-telemetry branch July 11, 2026 21:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant