feat(init): add init command for guided Sentry project setup

[betegon]

Summary

Adds sentry init — an AI-powered wizard that walks users through adding Sentry to their project. It detects the platform, installs the SDK, instruments the code, and configures error monitoring, tracing, and session replay.

Changes

Core wizard

• New init command backed by a Mastra AI workflow (hosted at getsentry/cli-init-api) that handles platform detection, SDK installation, and code instrumentation
• ASCII banner, AI transparency note, and review reminder in the wizard UX
• Tracing: unique trace IDs per wizard run with flattened span hierarchy
• Python platforms use venv for isolated dependency installation
• Magic values extracted into named constants (constants.ts)
• Docs page added to cli.sentry.dev

Security hardening

• Shell metacharacter blocklist: () subshell bypass, > < & redirection/background, $ ' " \ expansion/escaping, { } * ? glob/brace expansion, # shell comment
• Environment variable injection blocking (VAR=value cmd pattern)
• Remote-supplied cwd validation against project directory
• Dangerous executable blocklist and path-traversal prevention
• File descriptor cleanup on readSync failure
• Cross-platform shell execution ({ shell: true } instead of hardcoded sh)

Performance

• Pre-computed directory listing sent with first API call to skip initial list-dir round-trip
• _prevPhases cross-phase caching to reuse results across workflow steps (perf(init): pre-compute dir listing and send _prevPhases for cross-phase caching #307)

Testing & CI

• Wizard-runner unit tests (coverage 5.94% → 99.42%)
• Banner extracted to src/lib/banner.ts to break circular import
• Mock isolation fixes (spyOn instead of mock.module where possible)
• Simplified coverage pipeline (removed merge-lcov workaround)

Eval suite

The eval suite validates that the wizard produces correct, buildable Sentry instrumentation for each supported platform. It uses a 3-phase test architecture:

Phase 1: Wizard run

Each test scaffolds a fresh project from a platform template, then runs the full sentry init wizard against it. The wizard output (exit code, stdout/stderr, git diff, new files) is captured for the next phases.

Phase 2: Hard assertions (deterministic)

Five code-based pass/fail checks that run without any LLM:

1. exit-code — wizard exits 0
2. sdk-installed — the Sentry SDK package appears in the dependency file (package.json / requirements.txt)
3. init-present — Sentry.init (or sentry_sdk.init) appears in changed or new files
4. no-placeholder-dsn — no leftover placeholder DSNs (___PUBLIC_DSN___, YOUR_DSN_HERE, etc.)
5. build-succeeds — npm run build / equivalent passes after the wizard's changes

Phase 3: LLM judge (per-feature)

For each feature (errors, tracing, replay, logs, profiling, etc.), an LLM judge scores correctness:

• Official Sentry docs are fetched as ground truth (URLs mapped in feature-docs.json)
• GPT-4o evaluates the wizard's diff + new files against the docs on 4 criteria: feature-initialized, correct-imports, no-syntax-errors, follows-docs
• Each criterion is scored pass/fail/unknown; the overall feature score must be >= 0.5

Platforms

6 platform templates are covered:

Platform	Template	SDK
Express	express/	@sentry/node
Next.js	nextjs/	@sentry/nextjs
SvelteKit	sveltekit/	@sentry/sveltekit
React + Vite	react-vite/	@sentry/react
Flask	python-flask/	sentry-sdk
FastAPI	python-fastapi/	sentry-sdk

Running

bun run test:init-eval          # all platforms

Requires SENTRY_AUTH_TOKEN, SENTRY_ORG, SENTRY_PROJECT, and optionally OPENAI_API_KEY (LLM judge is skipped without it).

Test Plan

• All init unit tests pass (124 tests across 7 files)
• bun run lint and bun run typecheck pass
• CI passes (unit tests, e2e, lint, typecheck, build)
• CI workflow for eval is tracked separately in Run init evals on CI #290

🤖 Generated with Claude Code

[github-actions]

Semver Impact of This PR

🟡 Minor (new features)

📋 Changelog Preview

This is how your changes will appear in the changelog.
Entries from this PR are highlighted with a left border (blockquote style).

---

New Features ✨

• (init) Add init command for guided Sentry project setup by betegon in #283

• Input hardening against agent hallucinations by BYK in #370
• Add response caching for read-only API calls by BYK in #330

Bug Fixes 🐛

• (docs) Remove double borders and fix column alignment on landing page tables by betegon in #369

---

_{🤖 This preview updates automatically when you update the PR.}

[github-actions]

Codecov Results 📊

✅ 104 passed | Total: 104 | Pass Rate: 100% | Execution Time: 0ms

📊 Comparison with Base Branch

Metric	Change
Total Tests	—
Passed Tests	—
Failed Tests	—
Skipped Tests	—

✨ No test changes detected

All tests are passing successfully.

✅ Patch coverage is 95.72%. Project has 4049 uncovered lines.
✅ Project coverage is 81.98%. Comparing base (base) to head (head).

Files with missing lines (7)

File	Patch %	Lines
local-ops.ts	92.89%	⚠️ 35 Missing
sentry-url-parser.ts	76.19%	⚠️ 25 Missing
app.ts	81.36%	⚠️ 22 Missing
create.ts	93.98%	⚠️ 15 Missing
wizard-runner.ts	95.42%	⚠️ 12 Missing
help.ts	96.81%	⚠️ 3 Missing
utils.ts	88.00%	⚠️ 3 Missing

Coverage diff

@@            Coverage Diff             @@
##          main       #PR       +/-##
==========================================
+ Coverage    80.79%    81.98%    +1.19%
==========================================
  Files          132       140        +8
  Lines        21382     22473     +1091
  Branches         0         0         —
==========================================
+ Hits         17274     18424     +1150
- Misses        4108      4049       -59
- Partials         0         0         —

---

Generated by Codecov Action

[cursor]

Cursor Bugbot has reviewed your changes and found 1 potential issue.

^{Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

[cursor]

Cursor Bugbot has reviewed your changes and found 2 potential issues.

^{Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

[cursor]

Cursor Bugbot has reviewed your changes and found 3 potential issues.

^{Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

[cursor]

Cursor Bugbot has reviewed your changes and found 1 potential issue.

^{Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

[cursor]

Cursor Bugbot has reviewed your changes and found 2 potential issues.

Bugbot Autofix prepared fixes for both issues found in the latest run.

• ✅ Fixed: Self-hosted getOrgBaseUrl silently drops org slug
  • Added explicit error throw for self-hosted instances to prevent silent failure when org slug would be dropped.
• ✅ Fixed: Eval judge score zero when all criteria unknown
  • Changed default score from 0 to 1.0 when no criteria are gradable, treating inconclusive judge as pass rather than fail.

Or push these changes by commenting:

@cursor push 83acd8e5a2

Preview (83acd8e5a2)

diff --git a/src/lib/sentry-urls.ts b/src/lib/sentry-urls.ts
--- a/src/lib/sentry-urls.ts
+++ b/src/lib/sentry-urls.ts
@@ -21,11 +21,14 @@
  *
  * @param orgSlug - Organization slug
  * @returns Origin URL with org as subdomain
+ * @throws Error if called for self-hosted instances (subdomain pattern only works for SaaS)
  */
 export function getOrgBaseUrl(orgSlug: string): string {
   const base = getSentryBaseUrl();
   if (!isSentrySaasUrl(base)) {
-    return base;
+    throw new Error(
+      `getOrgBaseUrl() only supports Sentry SaaS URLs. For self-hosted instances, use path-based URLs with /organizations/${orgSlug}/`
+    );
   }
   const parsed = new URL(base);
   parsed.hostname = `${orgSlug}.${parsed.hostname}`;

diff --git a/test/init-eval/helpers/judge.ts b/test/init-eval/helpers/judge.ts
--- a/test/init-eval/helpers/judge.ts
+++ b/test/init-eval/helpers/judge.ts
@@ -119,7 +119,7 @@
   const gradable = parsed.criteria.filter((c) => c.pass !== "unknown");
   const passing = gradable.filter((c) => c.pass === true).length;
   const total = gradable.length;
-  const score = total > 0 ? passing / total : 0;
+  const score = total > 0 ? passing / total : 1.0;
 
   const verdict: JudgeVerdict = {
     criteria: parsed.criteria,

_{This Bugbot Autofix run was free. To enable autofix for future PRs, go to the Cursor dashboard.}

[cursor]

Cursor Bugbot has reviewed your changes and found 1 potential issue.

^{Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

[cursor]

Cursor Bugbot has reviewed your changes and found 2 potential issues.

^{Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

[cursor]

Redundant SaaS check in every URL construction call

Low Severity

Every URL-building function (e.g. buildOrgUrl, buildProjectUrl) calls isSaaS() which internally calls getSentryBaseUrl() + isSentrySaasUrl(), and then calls getOrgBaseUrl() which internally calls getSentryBaseUrl() + isSentrySaasUrl() again. This means each URL construction makes two redundant calls to read the environment variable and validate the hostname. The getOrgBaseUrl function could accept the already-resolved base URL as a parameter, or the SaaS check result could be passed through.

Additional Locations (1)

• src/lib/sentry-urls.ts#L24-L33

 

[cursor]

Empty error string shows blank message to user

Low Severity

formatError uses ?? (nullish coalescing) for the error message fallback chain: result.error ?? inner?.message ?? "Wizard failed...". If the server returns error: "" (empty string), ?? does not treat it as nullish, so message becomes "" and log.error("") displays a blank error line. Using || instead of the first ?? would fall through to the next alternative for empty strings.

 

[BYK]

I think we are in a good place, I don't think any of my comments are blockers to get this out of the door with a huge "experimental" banner.

One thing I noticed is we don't check if we are operating in a git repo, whether it is clear and safe for us to do things and roll back etc. I remember sentry-wizard having some code for this so I think we can just steal that for increased safety.

Once we are out of experimental mode, we should definitely re-evaluate the safety of our generic command call helper. On Linux systems, we might be able to easily jail that if we want to keep using generic bash commands.

[BYK]

This is a bit weird as we may want to extend existing Sentry installation (only errors to errors + tracing etc)

[BYK]

Uses a remote workflow that coordinates local file operations through the CLI.

Do we really need to call this out in the help text?

[BYK]

As mentioned earlier, I vote to remove this.

[BYK]

I'd also allow + and (for quoted args) for defensive parsing. We may also consider allowing multiple --features or --feature flags to join them. Similar to what we do with --field in the api command

[BYK]

Suggested change

	cancel(`Setup cancelled. Visit ${SENTRY_DOCS_URL} to set up manually.`);
	cancel(`Setup cancelled. You can visit ${SENTRY_DOCS_URL} to set up manually.`);

[BYK]

use log.error()

[BYK]

Why to stderr?

[BYK]

Why is the banner printing part not part of the intro helper already?

[BYK]

Okay, what? What is the sentry-cli tag? What are we doing here with a custom trace id etc?

[BYK]

I'd say if we can, we should support resuming from suspension from information stored on disk in case of an unexpected process termination