Skip to content

Require preset-usage README in community preset submission workflow (validate Spec Kit CLI syntax) #3103

Description

@mnriem

Problem

The automated community-preset submission workflow currently validates only that a README.md exists at the repository root. It does not validate that the linked README actually explains how to use the preset itself.

This passes trivially for monorepos. Example surfaced by #3102 (dfirs1car1o/sicario-spec): the catalog documentation field points at the repo-root README, which is a product pitch for the broader "SicarioSpec" framework / sicario CLI — not a guide to the sicario-core preset. The actual preset-usage doc lives at presets/sicario-core/README.md and is never the thing validated or linked.

Note: keeping the usage README outside the release ZIP is intentional and correct — it should be discoverable before a user downloads the artifact. The flaw is the shallowness of the check, not the README's location.

Proposal

Tighten the submission/validation workflow to require a preset-scoped usage README and validate its contents:

  1. Validate the README that documentation points to, not merely "a file named README at repo root." If documentation resolves to a preset subdir README, validate that file; if it points at a generic root README in a monorepo, flag it.

  2. Require usage instructions that reflect Spec Kit CLI / command syntax. At minimum, assert the linked README contains at least one valid specify preset ... invocation — ideally the catalog-install form matching the entry's download_url, e.g.:

    • specify preset add --from https://github.com/<org>/<repo>/releases/download/vX.Y.Z/<id>-X.Y.Z.zip
    • and/or specify preset add <id> / specify preset add --dev <path>

    This is the cleanest signal that distinguishes a genuine preset usage doc from a product pitch, and it is mechanically checkable.

  3. Recommend submitters set documentation to the preset-scoped README rather than repo root, so the catalog surfaces usage instead of marketing.

Suggested minimum README content (enforceable as structure)

  • What the preset does / what it provides
  • Install command using Spec Kit CLI syntax (see above)
  • When to use / when not to use

Caveat

"Does the README explain this preset" is partly a content judgment. The workflow can enforce structure (required sections, a valid specify preset ... snippet present, correct download URL) but a human reviewer should still sanity-check that the linked doc isn't just a funnel to a separate product/CLI.

Relationship to #3102

#3102 itself submits a legitimate true preset (the v0.4.0 ZIP is a clean preset.yml + 5 template overrides). The fix there is link hygiene (point documentation at the preset README). This issue is the systemic follow-up to harden the workflow so future monorepo submissions can't pass the README check with a product pitch.

Acceptance criteria

  • Workflow validates the README referenced by documentation (resolving to the preset-scoped doc), not just root presence
  • Workflow fails/flags submissions whose linked README lacks a valid specify preset ... invocation
  • presets/PUBLISHING.md updated to document the stricter README + CLI-syntax requirement
  • Reviewer guidance notes the residual human content check

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions