Security Audit #123

Workflow file for this run

.github/workflows/security-audit.yml at 6ce88d2

	name: Security Audit

	# Runs `npm audit` (and `bun audit` if available) on every PR and weekly.
	# Fails the workflow on high or critical vulnerabilities.
	#
	# Also runs `pip-audit` over every Python requirements file, `gitleaks` over the
	# working tree, and CodeQL SAST over the Python and JavaScript/TypeScript source
	# surfaces. Python dependencies and sub-critical SAST remain REPORTING ONLY
	# today -- see each job's posture note for its measured baseline or promotion
	# path. Secret scanning blocks every finding outside the exact reviewed-
	# fingerprint baseline. CodeQL findings with a security severity of 9.0 or
	# greater fail after the first exact-SHA critical baseline was triaged.
	#
	# SAST SCOPE: CodeQL does not support Bash, so this job does not imply coverage
	# of the 32k-line shell CLI. It does cover the shipped Python dashboard/runtime
	# and JavaScript/TypeScript CLI/dashboard surfaces. Saying that boundary here is
	# load-bearing: a green CodeQL job is evidence those supported languages were
	# analyzed, never evidence that every executable byte in this repository was.

	on:
	pull_request:
	branches: [main]
	# A VERSION push is a release. release.yml's required-ci job will not let a
	# release publish until this workflow reports success AT THAT EXACT SHA, and
	# it can only report if it actually runs there -- on pull_request and a
	# weekly cron alone, a release commit has no Security Audit run at all, and
	# requiring one would deadlock every release instead of gating it.
	push:
	paths:
	- 'VERSION'
	branches:
	- main
	schedule:
	# Mondays at 07:00 UTC
	- cron: '0 7 * * 1'
	workflow_dispatch:

	permissions:
	contents: read
	security-events: write

	concurrency:
	group: ${{ github.workflow }}-${{ github.ref }}
	cancel-in-progress: true

	jobs:
	npm-audit:
	name: npm audit (high+)
	runs-on: ubuntu-latest
	timeout-minutes: 10
	steps:
	- name: Checkout
	uses: actions/checkout@v4

	- name: Setup Node.js
	uses: actions/setup-node@v4
	with:
	node-version: '22'

	- name: Install dependencies (with lockfile so npm audit can apply package.json overrides)
	run: npm install

	- name: npm audit (production deps, high+)
	# audit-check.sh mirrors `npm audit --omit=dev --audit-level=high` but
	# waives documented, not-reachable advisories (see the script header for
	# each accepted GHSA + its rationale). A NEW high advisory that is not on
	# the allowlist still fails this gate -- it is a targeted exception, never
	# a blanket suppression. Kept in lockstep with scripts/local-ci.sh (both
	# call the same script) so the local pre-push gate and this CI gate agree.
	run: bash scripts/audit-check.sh

	python-audit:
	name: pip-audit (Python deps, reporting)
	runs-on: ubuntu-latest
	timeout-minutes: 15
	steps:
	- name: Checkout
	uses: actions/checkout@v4

	- name: Setup Python
	uses: actions/setup-python@v5
	with:
	python-version: '3.12'

	- name: Install pip-audit (pinned)
	run: python -m pip install --disable-pip-version-check 'pip-audit==2.9.0'

	- name: pip-audit every requirements file
	# POSTURE: REPORTING, NOT BLOCKING -- and this is a measured decision.
	#
	# Baseline on 2026-08-08, before this job existed, two advisories, BOTH
	# in files that ship to users via package.json files[]:
	# mcp/requirements.txt chromadb 1.5.9 PYSEC-2026-311 no fix version
	# web-app/requirements.txt ecdsa 0.19.2 PYSEC-2026-1325 no fix version
	# Neither has a fix release available, so a blocking gate would have
	# nothing to bump to. It would just be red forever.
	#
	# That matters more than usual HERE: release.yml's required-ci job waits
	# on this WORKFLOW's conclusion by name ("Security Audit"), not on named
	# jobs. Any job in this file that fails blocks EVERY release. Landing a
	# blocking scanner on an unmeasured -- or in this case known-dirty --
	# baseline would wedge the release pipeline, not tighten it.
	#
	# PROMOTION PATH, so this stays a plan and not a permanent excuse:
	# when both advisories above are resolved (upstream fix, or a pin we can
	# move to), flip the shipped files to blocking by deleting the
	# `continue-on-error` below for the shipped set and failing when their
	# JSON reports a non-empty `vulns` array. Waive anything intentionally
	# accepted BY ID with `--ignore-vuln <ID>` plus a written rationale,
	# exactly as scripts/audit-check.sh does for npm GHSAs. That mirrors the
	# house style: fail on any NEW advisory, waive documented ones by ID.
	#
	# WHY NOT "block only on HIGH/CRITICAL": pip-audit 2.9.0 has no severity
	# filter (no --severity flag) and its JSON carries no severity field --
	# each vuln is {id, fix_versions, aliases, description}. A severity
	# filter here would be a JSON query matching nothing, i.e. a gate that
	# looks strict and silently never fires. "Any advisory, waivable by ID"
	# is the honest translation and is what the promotion path uses.
	#
	# SHIPPED vs TEST-ONLY is recorded per file below and carried into the
	# JSON filenames, so the promotion above can flip the two sets
	# independently. Membership is from package.json files[]: dashboard/,
	# mcp/ and web-app/requirements.txt ship; the two *-test.txt do not.
	continue-on-error: true
	run: \|
	set -uo pipefail
	mkdir -p /tmp/pip-audit

	# FAIL CLOSED on a missing tool. A scanner that silently no-ops when
	# its binary is absent is worse than no scanner, because the green
	# check implies coverage that never happened -- see sbom.yml:7-23 for
	# the workflow in this repo that ran green for months on a dead
	# trigger. This step is `continue-on-error` for FINDINGS; a tool that
	# is not installed is not a finding, it is an absent measurement, so
	# it exits nonzero here and the assert step below also fails hard.
	if ! command -v pip-audit >/dev/null 2>&1; then
	echo "FAIL: pip-audit is not installed -- audit did NOT run"
	exit 1
	fi

	# Each file is audited INDIVIDUALLY rather than with one multi-flag
	# invocation, so a per-file JSON exists to assert on and the
	# shipped/test posture can diverge later without restructuring.
	# SHIPPED (in package.json files[], reaches every install):
	for f in dashboard/requirements.txt mcp/requirements.txt web-app/requirements.txt; do
	echo "::group::pip-audit (shipped) $f"
	pip-audit -r "$f" --format json \
	--output "/tmp/pip-audit/shipped-$(echo "$f" \| tr '/' '_').json" \
	--progress-spinner off \|\| true
	echo "::endgroup::"
	done
	# TEST-ONLY (not in files[], does not reach users):
	for f in requirements-test.txt web-app/requirements-test.txt; do
	echo "::group::pip-audit (test-only) $f"
	pip-audit -r "$f" --format json \
	--output "/tmp/pip-audit/testonly-$(echo "$f" \| tr '/' '_').json" \
	--progress-spinner off \|\| true
	echo "::endgroup::"
	done

	- name: Assert the audit actually produced a report for every file
	# THE FAIL-CLOSED GATE. The step above is continue-on-error so that
	# FINDINGS report instead of blocking. That alone would also swallow a
	# crashed or missing scanner, which is the exact sbom.yml failure mode.
	# So correctness is asserted on the ARTIFACT, never on an exit code:
	# each expected JSON must exist AND parse. An empty result is not
	# evidence of a clean tree, it is an absent measurement.
	run: \|
	set -euo pipefail
	missing=0
	# Enumerated individually and NOT counted: a count cannot say WHICH
	# file was dropped, and would pick up slack it was never meant to have.
	for want in \
	shipped-dashboard_requirements.txt.json \
	shipped-mcp_requirements.txt.json \
	shipped-web-app_requirements.txt.json \
	testonly-requirements-test.txt.json \
	testonly-web-app_requirements-test.txt.json ; do
	p="/tmp/pip-audit/$want"
	if [ ! -s "$p" ]; then
	echo "FAIL: no pip-audit report for $want -- that file was NOT audited"
	missing=1
	continue
	fi
	python -c "import json,sys; json.load(open(sys.argv[1]))" "$p" \|\| {
	echo "FAIL: pip-audit report $want is not valid JSON -- the scan did not complete"
	missing=1
	}
	done
	[ "$missing" -eq 0 ] \|\| exit 1
	echo "all five requirements files produced a parseable pip-audit report"

	- name: Summarise findings (reporting)
	if: always()
	run: \|
	set -uo pipefail
	python - <<'PY'
	import glob, json, os
	for p in sorted(glob.glob('/tmp/pip-audit/*.json')):
	try:
	d = json.load(open(p))
	except Exception:
	continue
	hits = [(x['name'], v['id'])
	for x in d.get('dependencies', []) for v in (x.get('vulns') or [])]
	if hits:
	tag = 'SHIPPED' if os.path.basename(p).startswith('shipped-') else 'test-only'
	for name, vid in hits:
	print(f"::warning::[{tag}] {os.path.basename(p)}: {name} {vid}")
	PY

	- name: Upload pip-audit reports
	if: always()
	uses: actions/upload-artifact@v4
	with:
	name: pip-audit-reports
	path: /tmp/pip-audit/
	if-no-files-found: error

	secret-scan:
	name: gitleaks (secrets, new-findings-blocking)
	runs-on: ubuntu-latest
	timeout-minutes: 15
	steps:
	- name: Checkout
	uses: actions/checkout@v4

	- name: Install gitleaks (pinned)
	run: \|
	set -euo pipefail
	curl -sSfL -o /tmp/gitleaks.tar.gz \
	https://github.com/gitleaks/gitleaks/releases/download/v8.30.0/gitleaks_8.30.0_linux_x64.tar.gz
	tar -xzf /tmp/gitleaks.tar.gz -C /tmp gitleaks
	/tmp/gitleaks version

	- name: gitleaks scan (working tree)
	# POSTURE: NEW FINDINGS BLOCK. A fresh pinned v8.30.0 scan on 2026-08-17
	# found 35 current-tree hits: three copies of the same public PostHog
	# ingestion key, two configuration/prose false positives, and 30
	# intentionally secret-shaped scanner/redaction test fixtures. Every hit
	# received an exact file/rule/line fingerprint in .gitleaksignore. There
	# are no blanket path or rule suppressions, so moving or changing one of
	# those lines, or adding any other finding, returns the scanner's default
	# nonzero exit and blocks this workflow and the release gate.
	#
	# `dir` is the working-tree subcommand in v8.30.0; the older
	# `detect --no-git` spelling no longer exists in this version.
	run: \|
	set -euo pipefail
	# FAIL CLOSED: an absent binary is not "no secrets found".
	if [ ! -x /tmp/gitleaks ]; then
	echo "FAIL: gitleaks is not installed -- secret scan did NOT run"
	exit 1
	fi
	/tmp/gitleaks dir . \
	--gitleaks-ignore-path .gitleaksignore \
	--report-format json \
	--report-path /tmp/gitleaks-report.json \
	--redact \
	--no-banner

	- name: Assert the secret scan completed cleanly
	# Defense in depth: the scanner already exits nonzero on an unmatched
	# finding. Also require a parseable empty report so a changed tool that
	# returns zero while reporting findings cannot silently weaken the gate.
	run: \|
	set -euo pipefail
	if [ ! -f /tmp/gitleaks-report.json ]; then
	echo "FAIL: gitleaks produced no report -- the tree was NOT scanned"
	exit 1
	fi
	n=$(python3 -c "import json,sys; print(len(json.load(open('/tmp/gitleaks-report.json'))))")
	if [ "$n" -ne 0 ]; then
	echo "FAIL: gitleaks reported $n unmatched finding(s)"
	exit 1
	fi
	echo "gitleaks reported zero unmatched findings"

	- name: Upload gitleaks report
	if: always()
	uses: actions/upload-artifact@v4
	with:
	name: gitleaks-report
	path: /tmp/gitleaks-report.json
	if-no-files-found: error

	sast:
	name: CodeQL SAST (${{ matrix.language }}, critical-blocking)
	runs-on: ubuntu-latest
	timeout-minutes: 20
	strategy:
	fail-fast: false
	matrix:
	# CodeQL's language identifier deliberately combines JavaScript and
	# TypeScript. Python is a separate database. Bash is unsupported and is
	# explicitly excluded in the workflow header rather than silently
	# treated as covered.
	language: [javascript-typescript, python]
	steps:
	- name: Checkout
	uses: actions/checkout@v4

	- name: Initialize CodeQL
	# Pinned to the commit behind codeql-action v3 on 2026-08-13. A moving
	# major tag can change query behavior without a repository commit and
	# make a measured baseline incomparable to its successor.
	uses: github/codeql-action/init@f3712979fa5f215279b101dd0a2e3bdfb4353324
	with:
	languages: ${{ matrix.language }}

	- name: Analyze supported source
	id: analyze
	# Analysis/upload failures block. Findings below the critical threshold
	# still report in code scanning and retained SARIF; the separately
	# measured gate below rejects security-severity >= 9.0 after all seven
	# first-baseline critical candidates received reviewed dispositions.
	uses: github/codeql-action/analyze@f3712979fa5f215279b101dd0a2e3bdfb4353324
	with:
	category: /language:${{ matrix.language }}
	output: /tmp/codeql-results
	upload: always

	- name: Assert CodeQL SARIF and reject unreviewed critical findings
	# FAIL CLOSED ON THE ARTIFACT. A completed action without a retained,
	# parseable result is not evidence that analysis happened. Validate the
	# action-provided output path rather than assuming a generated filename.
	env:
	SARIF_OUTPUT: ${{ steps.analyze.outputs.sarif-output }}
	run: \|
	set -euo pipefail
	test -n "$SARIF_OUTPUT"
	test -d "$SARIF_OUTPUT"
	python3 - "$SARIF_OUTPUT" <<'PY'
	import glob
	import json
	import os
	import sys

	root = sys.argv[1]
	reports = sorted(glob.glob(os.path.join(root, "*", ".sarif"), recursive=True))
	if not reports:
	raise SystemExit("FAIL: CodeQL produced no SARIF report")
	# These exact identities are the seven first-baseline findings reviewed
	# in 09184232. They remain visible in SARIF/code scanning. The blocking
	# decision accepts only the same rule, path, and stable source-line
	# fingerprint, so moving to a different sink or changing the sink line
	# creates an unreviewed critical finding and fails closed.
	reviewed_critical = {
	("py/command-line-injection", "api-examples/python-api.py", "d3e00d2bc3c0077d:1"),
	("py/command-line-injection", "dashboard/api_releases.py", "9f905b10583ce31a:1"),
	("py/command-line-injection", "dashboard/control.py", "1cd558210b41452d:1"),
	("py/command-line-injection", "dashboard/server.py", "74cce5fbedefb794:1"),
	("py/command-line-injection", "dashboard/server.py", "1cd558210b41452d:1"),
	("py/command-line-injection", "web-app/server.py", "2b985af458631366:1"),
	("py/command-line-injection", "web-app/server.py", "97626c01411ab378:1"),
	}
	seen_critical = set()
	reviewed = []
	unreviewed = []
	for report in reports:
	with open(report, encoding="utf-8") as handle:
	payload = json.load(handle)
	if payload.get("version") != "2.1.0" or not isinstance(payload.get("runs"), list):
	raise SystemExit(f"FAIL: invalid SARIF structure: {report}")
	for run in payload["runs"]:
	tool = run.get("tool") or {}
	rules = []
	rules.extend(((tool.get("driver") or {}).get("rules") or []))
	for extension in tool.get("extensions") or []:
	rules.extend(extension.get("rules") or [])
	severities = {}
	for rule in rules:
	raw = (rule.get("properties") or {}).get("security-severity")
	try:
	severities[rule.get("id")] = float(raw)
	except (TypeError, ValueError):
	continue
	for result in run.get("results") or []:
	rule_id = result.get("ruleId")
	severity = severities.get(rule_id, 0.0)
	if severity < 9.0:
	continue
	location = ((result.get("locations") or [{}])[0].get("physicalLocation") or {})
	artifact = (location.get("artifactLocation") or {}).get("uri", "unknown")
	line = (location.get("region") or {}).get("startLine", "?")
	fingerprint = (result.get("partialFingerprints") or {}).get(
	"primaryLocationLineHash"
	)
	identity = (rule_id, artifact, fingerprint)
	finding = (rule_id, severity, artifact, line, fingerprint)
	if identity in seen_critical:
	unreviewed.append(finding)
	continue
	seen_critical.add(identity)
	if identity in reviewed_critical:
	reviewed.append(finding)
	else:
	unreviewed.append(finding)
	print(f"validated {len(reports)} CodeQL SARIF report(s)")
	for rule_id, severity, artifact, line, fingerprint in reviewed:
	print(
	f"REVIEWED CRITICAL: {rule_id} severity={severity:g} "
	f"at {artifact}:{line} fingerprint={fingerprint}"
	)
	if unreviewed:
	for rule_id, severity, artifact, line, fingerprint in unreviewed:
	print(
	f"UNREVIEWED CRITICAL: {rule_id} severity={severity:g} "
	f"at {artifact}:{line} fingerprint={fingerprint}"
	)
	raise SystemExit(
	f"FAIL: CodeQL reported {len(unreviewed)} unreviewed or duplicate "
	"finding(s) at security-severity >= 9.0"
	)
	print(
	"CodeQL critical gate clean "
	f"({len(reviewed)} exact reviewed finding(s), 0 unreviewed)"
	)
	PY

	- name: Retain CodeQL SARIF
	if: always()
	uses: actions/upload-artifact@v4
	with:
	name: codeql-sarif-${{ matrix.language }}
	path: /tmp/codeql-results/
	if-no-files-found: error

	bun-audit:
	name: bun audit (best-effort)
	runs-on: ubuntu-latest
	timeout-minutes: 10
	steps:
	- name: Checkout
	uses: actions/checkout@v4

	- name: Setup Bun
	uses: oven-sh/setup-bun@v2
	with:
	bun-version: 1.3.13

	- name: Probe for `bun audit` subcommand
	id: probe
	run: \|
	set +e
	bun audit --help >/dev/null 2>&1
	ec=$?
	set -e
	if [ "$ec" -eq 0 ]; then
	echo "supported=true" >> "$GITHUB_OUTPUT"
	else
	echo "supported=false" >> "$GITHUB_OUTPUT"
	echo "::notice::`bun audit` not available in bun v1.3.13 -- skipping"
	fi

	- name: Install loki-ts dependencies
	if: steps.probe.outputs.supported == 'true'
	working-directory: loki-ts
	run: bun install

	- name: Run bun audit
	if: steps.probe.outputs.supported == 'true'
	working-directory: loki-ts
	run: \|
	set +e
	bun audit
	ec=$?
	set -e
	if [ "$ec" -ne 0 ]; then
	# ADVISORY, matching this job's own name. The AUTHORITATIVE gate is
	# the sibling `npm audit (high+)` job, which runs scripts/audit-check.sh
	# and applies the documented not-reachable waiver list -- a NEW high
	# advisory still fails there and still blocks the release.
	#
	# This step hard-failed while being labelled best-effort, which made
	# it a blocking gate with no waiver mechanism. It fired on
	# GHSA-v2hh-gcrm-f6hx in fast-uri, which is four levels deep inside a
	# dependency we do not control:
	# loki-mode -> @anthropic-ai/claude-agent-sdk -> @modelcontextprotocol/sdk
	# -> ajv -> fast-uri
	# There is no action available to us: no direct dependency to bump,
	# and the fix has to land upstream. Blocking every release on an
	# unactionable transitive advisory trains people to ignore the gate,
	# which is strictly worse than reporting it.
	echo "::warning::bun audit reported vulnerabilities (advisory; the npm audit job is the gate)"
	else
	echo "bun audit clean"
	fi

Navigation Menu

Security Audit #123

Workflow file

Security Audit #123

Uh oh!

Workflow file for this run