feat(harness): agentic-first context & harness-integrity system

[TeddyCr]

Summary

Restructures the repo's agent-facing context into a layered, self-maintaining system so coding agents (Claude Code, Codex) load the right guidance at the right time — and adds CI that warns when
that guidance drifts from the code. No product/runtime code changes: entirely agent context, docs, tooling, and non-blocking CI.

Motivation

CLAUDE.md had grown into a single ~700-line always-loaded file mixing always-true rules with language/path-specific detail, and nothing caught it drifting from the code. This PR splits
always-loaded guidance from path-scoped rules that auto-load only when you touch a matching file, gives agents a discoverable map of existing docs instead of re-deriving from code, and adds a
warnings-only CI check so dead references / size budgets / broken globs surface on PRs.

What changed

1. CLAUDE.md → thin map + path-scoped rules — trimmed to a 169-line always-loaded map; nine auto-loading .claude/rules/*.md (java, frontend-react, frontend-styling, component-library, i18n,
   frontend-playwright, python-ingestion, schema-first, migrations), each with a paths: glob.

2. System & knowledge maps — ARCHITECTURE.md (system map + invariants), docs/index.md (doc index with CURRENT/STALE/SUPERSEDED freshness verified against the tree), docs/design-patterns.md (GoF
   patterns idiomatic to this repo).

3. Deterministic reference-doc generators — scripts/generate_entity_index.py → docs/generated/entity-index.md (81 entities); scripts/generate_api_reference.py → docs/generated/api-reference.md
   (1748 endpoints from JAX-RS annotations). Byte-identical output; make generate-reference-docs.

4. Harness-integrity CI (warnings-only) — scripts/harness/check_harness.py (6 checks: dead refs, AGENTS↔CLAUDE sync, skill symlinks, doc-size budgets, rule globs, generated-doc freshness) +
   .github/workflows/harness-integrity.yml. Intentionally non-blocking.

5. AGENTS.md ↔ CLAUDE.md sync — AGENTS.md is now a git symlink to CLAUDE.md (was a drifting 255-line mirror).

6. Enforcement hooks + skills — .claude/settings.json PreToolUse hooks block edits to generated output/workflows; .pre-commit-config.yaml + scripts/check-ui-*.sh; skills consolidated into a single
   skills/ source of truth (symlinked from .claude/skills, .agents/skills).

7. Repo audit docs — docs/tech-debt.md, docs/quality.md, docs/golden-principles.md (evidence-cited). docs/harness-audit/ left intentionally untracked.

Verification

• check_harness.py → only the two standing warnings, no new ones.
• Generators byte-stable: make generate-reference-docs leaves git diff empty (across PYTHONHASHSEED 0/1/999).
• CLAUDE.md = 169 lines (under 200); AGENTS.md resolves to identical content via the symlink.

[Copilot]

Copilot wasn't able to review this pull request because it exceeds the maximum number of files (300). Try reducing the number of changed files and requesting a review from Copilot again.

[gitar-bot]

Code Review 👍 Approved with suggestions 0 resolved / 3 findings

Introduces an agentic-first context and harness-integrity system to structure agent guidance and validate documentation freshness. Consider addressing minor findings regarding unclosed file handles in check_harness.py, working tree mutation during generation checks, and silent failures in the freshness check.

💡 Bug: generated-fresh check silently passes if a generator fails

📄 scripts/harness/check_harness.py:440-453

check_generated_fresh runs each generator with subprocess.run(..., check=False) and then only inspects git diff on docs/generated. If a generator errors out (import error, exception, missing dep) it writes nothing, the diff is empty, and the freshness check reports clean — the exact opposite of its purpose. Since this is the integrity tool meant to catch doc drift, a crashing generator would go unnoticed. Capture the return code and emit a warning when a generator exits non-zero.

Warn when a generator exits non-zero so a crash isn't mistaken for 'fresh'.

for gen in generators:
    res = subprocess.run([sys.executable, rp(gen)], cwd=REPO,
                         stdout=subprocess.DEVNULL, stderr=subprocess.PIPE, text=True, check=False)
    if res.returncode != 0:
        warnings.append(Warn("generated-fresh", gen, 1,
                             f"generator failed (exit {res.returncode}); freshness unverified"))

💡 Quality: harness 'check' mutates the working tree as a side effect

📄 scripts/harness/check_harness.py:440-453

check_generated_fresh regenerates docs/generated on every invocation, so running make harness-check locally can leave uncommitted regenerated files behind — surprising for something named a 'check'. Consider regenerating into a temp dir and diffing, or restoring the tree afterward, so the check is read-only. Minor since the CI job is non-blocking, but it can confuse local runs and pollute an in-progress working tree.

💡 Quality: Unclosed file handles in check_harness.py helpers

📄 scripts/harness/check_harness.py:204-206 📄 scripts/harness/check_harness.py:371

maven_plugin_prefixes() and check_skill_symlinks() open files with bare open(...).read() without a context manager (lines 205 and 371), leaking file descriptors during the tree walk. Use with open(...) as fh: (or the existing read() helper) for consistency and to avoid ResourceWarnings. Low impact as this is a short-lived script, but trivially fixable.

🤖 Prompt for agents

Code Review: Introduces an agentic-first context and harness-integrity system to structure agent guidance and validate documentation freshness. Consider addressing minor findings regarding unclosed file handles in check_harness.py, working tree mutation during generation checks, and silent failures in the freshness check.

1. 💡 Bug: generated-fresh check silently passes if a generator fails
   Files: scripts/harness/check_harness.py:440-453

   check_generated_fresh runs each generator with subprocess.run(..., check=False) and then only inspects `git diff` on docs/generated. If a generator errors out (import error, exception, missing dep) it writes nothing, the diff is empty, and the freshness check reports clean — the exact opposite of its purpose. Since this is the integrity tool meant to catch doc drift, a crashing generator would go unnoticed. Capture the return code and emit a warning when a generator exits non-zero.

   Fix (Warn when a generator exits non-zero so a crash isn't mistaken for 'fresh'.):
   for gen in generators:
       res = subprocess.run([sys.executable, rp(gen)], cwd=REPO,
                            stdout=subprocess.DEVNULL, stderr=subprocess.PIPE, text=True, check=False)
       if res.returncode != 0:
           warnings.append(Warn("generated-fresh", gen, 1,
                                f"generator failed (exit {res.returncode}); freshness unverified"))

2. 💡 Quality: harness 'check' mutates the working tree as a side effect
   Files: scripts/harness/check_harness.py:440-453

   check_generated_fresh regenerates docs/generated on every invocation, so running `make harness-check` locally can leave uncommitted regenerated files behind — surprising for something named a 'check'. Consider regenerating into a temp dir and diffing, or restoring the tree afterward, so the check is read-only. Minor since the CI job is non-blocking, but it can confuse local runs and pollute an in-progress working tree.

3. 💡 Quality: Unclosed file handles in check_harness.py helpers
   Files: scripts/harness/check_harness.py:204-206, scripts/harness/check_harness.py:371

   maven_plugin_prefixes() and check_skill_symlinks() open files with bare open(...).read() without a context manager (lines 205 and 371), leaking file descriptors during the tree walk. Use `with open(...) as fh:` (or the existing read() helper) for consistency and to avoid ResourceWarnings. Low impact as this is a short-lived script, but trivially fixable.

Options

Display: compact → Showing less information.

Comment with these commands to change the behavior for this request:

Compact

gitar display:verbose

_{Was this helpful? React with 👍 / 👎 | Gitar | Powered by Gitar — free for open source}

[github-actions]

Jest test Coverage

UI tests summary

Lines	Statements	Branches	Functions
	65.67% (76800/116936)	49.54% (46114/93066)	50.76% (13892/27365)

[sonarqubecloud]

Quality Gate passed for 'open-metadata-ui'

Issues
1216 New issues
0 Accepted issues

Measures
13 Security Hotspots
0.0% Coverage on New Code
3.0% Duplication on New Code

See analysis details on SonarQube Cloud