Require preset-usage README with Spec Kit CLI syntax in preset submissions (#3104)

* Require preset-usage README with Spec Kit CLI syntax in submissions Tighten the community preset submission workflow so it validates the README referenced by the documentation field rather than merely checking for a root README. The workflow now fails submissions whose linked README lacks a valid 'specify preset add ...' command and flags monorepo submissions that point documentation at a generic root README. - Add a required Documentation URL field to the preset issue template - Add validation step 2d (documentation README + CLI-syntax check) to .github/workflows/add-community-preset.md and recompile the lock file - Document the stricter usage-README requirement and reviewer content check in presets/PUBLISHING.md Closes #3103 Assisted-by: GitHub Copilot (model: Claude Opus 4.8, autonomous) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Align preset README docs with workflow's actual enforcement Address PR review feedback on #3104: - PUBLISHING.md: clarify that only README resolution + a valid 'specify preset add ...' command are mechanically enforced; the preset-scoped-README and minimum-structure items are reviewer expectations, not automated checks. - PUBLISHING.md: state that a missing 'specify preset add ...' command is a hard validation failure (check 2d), not just 'flagged for changes'. - preset_submission.yml: require 'specify preset add ...' (not the looser 'specify preset ...') to match the workflow validation. Assisted-by: GitHub Copilot (model: Claude Opus 4.8, autonomous) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Tighten preset README validation and docs per PR review Address PR review feedback on #3104: - Workflow Step 2c: drop the generic repo-root README.md check so the README requirement is enforced exactly once, in Step 2d, against the file the documentation field points to (avoids monorepo false-positive). - Workflow Step 2d: restrict the documentation URL to GitHub-hosted README URLs (github.com/.../blob/... or raw.githubusercontent.com/...) before fetching user-provided input. - PUBLISHING.md: add the required 'id' field to the example catalog entry. - preset_submission.yml: fix the Documentation URL placeholder to match the recommended monorepo presets/<id>/README.md pattern. - Recompile add-community-preset.lock.yml (body hash only). Assisted-by: GitHub Copilot (model: Claude Opus 4.8, autonomous) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Refine preset README validation rules per PR review Address PR review feedback on #3104: - Workflow Step 2d: broaden the documentation URL allowlist to also accept github.com/.../raw/... URLs; strip any fragment/query before fetching so the target is deterministic; clarify that a 'specify preset add --from <url>' command only counts when its URL matches the submitted Download URL (a different --from URL does not satisfy the requirement, though other accepted forms still can). - PUBLISHING.md: show both accepted download URL shapes (tag archive and release asset) in the README install example instead of implying only the releases/download form. - preset_submission.yml: remove the ambiguous generic 'README.md with description and usage instructions' checkbox; the linked-README requirement is the single source of truth. - Recompile add-community-preset.lock.yml (body hash only). Assisted-by: GitHub Copilot (model: Claude Opus 4.8, autonomous) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Clarify install-command requirement wording per PR review Address PR review feedback on #3104: the previous 'matching the download URL' wording overstated the requirement. Only the 'specify preset add --from <url>' form needs an exact download-URL match; other accepted forms ('specify preset add <id>' / '--dev <path>') don't reference the download URL at all. - preset_submission.yml: reword the Documentation URL description and the Submission Requirements checkbox to reflect what's enforced vs preferred. - PUBLISHING.md: clarify the reviewer note so the exact-match rule is scoped to the --from form. Assisted-by: GitHub Copilot (model: Claude Opus 4.8, autonomous) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Require README.md target and fix release-ZIP wording per PR review Address PR review feedback on #3104: - Workflow Step 2d: add an explicit check that the documentation URL path ends with README.md (case-insensitive) after stripping fragment/query, so a non-README markdown file is rejected before fetching. - PUBLISHING.md: reword the release-ZIP note, which conflicted with the earlier preset structure guidance. The real requirement is that the README is reachable at the documentation URL before download; it's fine for the same file to also ship inside the release ZIP. - Recompile add-community-preset.lock.yml (body hash only). Assisted-by: GitHub Copilot (model: Claude Opus 4.8, autonomous) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Use stable unnumbered anchor for Usage README Requirements Address PR review feedback on #3104: drop the '6.' prefix from the 'Usage README Requirements' heading so its GitHub anchor isn't tied to a section number (brittle under renumbering, and avoids confusion with the top-level 'Best Practices' TOC item). Update the Prerequisites cross-link to the new #usage-readme-requirements anchor. Assisted-by: GitHub Copilot (model: Claude Opus 4.8, autonomous) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Align README requirement wording with enforced checks per PR review Address PR review feedback on #3104: - PUBLISHING.md: the 'mechanically enforces' summary now lists all Step 2d checks (GitHub-hosted URL, path ends with README.md, resolves, contains a valid 'specify preset add ...' command), instead of only two. - PUBLISHING.md: reword the PR checklist item so a usage README + install command is the requirement, with preset-scoped README recommended for monorepos (matches the workflow's flag-not-fail behavior). - preset_submission.yml: include the full 'specify preset add' prefix on the --dev and --from forms in the field description and checklist so submitters copy the exact syntax. Assisted-by: GitHub Copilot (model: Claude Opus 4.8, autonomous) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Fix grammar in Usage README Requirements intro Address PR review feedback on #3104: remove the incorrect colon after 'the linked README' so the sentence reads naturally. Assisted-by: GitHub Copilot (model: Claude Opus 4.8, autonomous) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Avoid lossy raw URL rewrite for slash-containing refs per PR review Address PR review feedback on #3104: rewriting documentation URLs into the raw.githubusercontent.com/<owner>/<repo>/<ref>/<path> form can't reliably represent refs that contain slashes (e.g. a feature/foo branch). Step 2d now fetches github.com blob URLs by swapping only /blob/ -> /raw/, and fetches github.com/.../raw/... and raw.githubusercontent.com/... URLs as-is, instead of reconstructing the raw host form. Recompile add-community-preset.lock.yml (body hash only). Assisted-by: GitHub Copilot (model: Claude Opus 4.8, autonomous) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-07-03 12:28:06 +08:00 · 2026-06-24 17:06:51 -05:00
parent d6cddd4127
commit 96039d36d2
4 changed files with 125 additions and 9 deletions
--- a/.github/ISSUE_TEMPLATE/preset_submission.yml
+++ b/.github/ISSUE_TEMPLATE/preset_submission.yml
@@ -77,6 +77,18 @@ body:
    validations:
      required: true

+  - type: input
+    id: documentation
+    attributes:
+      label: Documentation URL
+      description: |
+        Link to the README that explains how to use **this preset** (not a general product/framework pitch).
+        Prefer the preset-scoped README (e.g. `presets/<id>/README.md` in a monorepo) over the repository root README.
+        It must contain at least one valid `specify preset add ...` install command — ideally `specify preset add --from <download-url>` using the exact Download URL above (other forms such as `specify preset add <preset-id>` or `specify preset add --dev <path>` are also accepted).
+      placeholder: "https://github.com/your-org/spec-kit-presets/blob/main/presets/your-preset/README.md"
+    validations:
+      required: true
+
  - type: input
    id: license
    attributes:
@@ -175,7 +187,7 @@ body:
      options:
        - label: Valid `preset.yml` manifest included
          required: true
-        - label: README.md with description and usage instructions
+        - label: Linked README (Documentation URL) explains how to use this preset and includes a valid `specify preset add ...` command (preferably `specify preset add --from <download-url>` using the exact download URL)
          required: true
        - label: LICENSE file included
          required: true
--- a/.github/workflows/add-community-preset.lock.yml
+++ b/.github/workflows/add-community-preset.lock.yml
@@ -1,4 +1,4 @@
-# gh-aw-metadata: {"schema_version":"v4","frontmatter_hash":"b4ba1db5fdec754fa825cc3160879924118bc454a781eed70ef6c90beab83a95","body_hash":"392ace500b7cb9b0aa6b020d150841de398bcbcfe54dbad729f0d860d698bde2","compiler_version":"v0.79.8","strict":true,"agent_id":"copilot","engine_versions":{"copilot":"1.0.60"}}
+# gh-aw-metadata: {"schema_version":"v4","frontmatter_hash":"b4ba1db5fdec754fa825cc3160879924118bc454a781eed70ef6c90beab83a95","body_hash":"cb6c19088fa13da0a8320c174e8c14c4887d2c8a005a5cb2d2d2faa3f890de39","compiler_version":"v0.79.8","strict":true,"agent_id":"copilot","engine_versions":{"copilot":"1.0.60"}}
 # gh-aw-manifest: {"version":1,"secrets":["COPILOT_GITHUB_TOKEN","GH_AW_CI_TRIGGER_TOKEN","GH_AW_GITHUB_MCP_SERVER_TOKEN","GH_AW_GITHUB_TOKEN","GITHUB_TOKEN"],"actions":[{"repo":"actions/checkout","sha":"df4cb1c069e1874edd31b4311f1884172cec0e10","version":"v6.0.3"},{"repo":"actions/download-artifact","sha":"3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c","version":"v8.0.1"},{"repo":"actions/github-script","sha":"3a2844b7e9c422d3c10d287c895573f7108da1b3","version":"v9.0.0"},{"repo":"actions/setup-node","sha":"48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e","version":"v6.4.0"},{"repo":"actions/upload-artifact","sha":"043fb46d1a93c77aae656e7c1c64a875d1fc6a0a","version":"v7.0.1"},{"repo":"github/gh-aw-actions/setup","sha":"c0338fef4749d08c21f8f975fb0e37efa17dda47","version":"v0.79.8"}],"containers":[{"image":"ghcr.io/github/gh-aw-firewall/agent:0.27.2","digest":"sha256:f88e5b17b6b7a600117bc121114d6ce2155c88c983c0c939c5df884f730fa1d6","pinned_image":"ghcr.io/github/gh-aw-firewall/agent:0.27.2@sha256:f88e5b17b6b7a600117bc121114d6ce2155c88c983c0c939c5df884f730fa1d6"},{"image":"ghcr.io/github/gh-aw-firewall/api-proxy:0.27.2","digest":"sha256:ee39841d980878ebbb87592903b06d31a1af500c71525c9616f7e8e2a27041a4","pinned_image":"ghcr.io/github/gh-aw-firewall/api-proxy:0.27.2@sha256:ee39841d980878ebbb87592903b06d31a1af500c71525c9616f7e8e2a27041a4"},{"image":"ghcr.io/github/gh-aw-firewall/squid:0.27.2","digest":"sha256:2e3a717e5f19a654cd9a2263beb52012b56bcb68562ec5ae2e42f9d156b49591","pinned_image":"ghcr.io/github/gh-aw-firewall/squid:0.27.2@sha256:2e3a717e5f19a654cd9a2263beb52012b56bcb68562ec5ae2e42f9d156b49591"},{"image":"ghcr.io/github/gh-aw-mcpg:v0.3.25","digest":"sha256:c10331ad17668ef89f38f5e356678788a40b0cd5fef96e8f92e1d9c1de47cbaa","pinned_image":"ghcr.io/github/gh-aw-mcpg:v0.3.25@sha256:c10331ad17668ef89f38f5e356678788a40b0cd5fef96e8f92e1d9c1de47cbaa"},{"image":"ghcr.io/github/github-mcp-server:v1.1.2","digest":"sha256:30197479d8036c7811892bc07e06f9a05c9ef3cdd79bc59f256d50647f95788c","pinned_image":"ghcr.io/github/github-mcp-server:v1.1.2@sha256:30197479d8036c7811892bc07e06f9a05c9ef3cdd79bc59f256d50647f95788c"}]}
 # This file was automatically generated by gh-aw (v0.79.8). DO NOT EDIT. To debug this workflow, load the skill at https://github.com/github/gh-aw/blob/main/debug.md
 #
--- a/.github/workflows/add-community-preset.md
+++ b/.github/workflows/add-community-preset.md
@@ -73,6 +73,7 @@ fields):
 | Author | `author` | Yes |
 | Repository URL | `repository` | Yes |
 | Download URL | `download-url` | Yes |
+| Documentation URL | `documentation` | Yes |
 | License | `license` | Yes |
 | Required Spec Kit Version | `speckit-version` | Yes |
 | Required Extensions | `required-extensions` | No |
@@ -100,17 +101,70 @@ deciding pass/fail:
 ### 2c. Repository validation
 - Fetch the repository URL — confirm it exists and is publicly accessible
 - Confirm the repository contains a `preset.yml` file
- Confirm the repository contains a `README.md` file
 - Confirm the repository contains a `LICENSE` file

-### 2d. Release and download URL validation
+> The README requirement is enforced once, in **Step 2d**, against the specific file the
+> `documentation` field points to — not a generic repository-root `README.md`. This avoids
+> the monorepo false-positive where a root README exists but isn't the preset-usage doc.
+
+### 2d. Documentation README validation
+
+The `documentation` field must point to the README that explains **how to use this
+preset** — not just any file named `README.md`, and not a product/framework pitch.
+
+- **Restrict the URL to GitHub before fetching.** The `documentation` value is
+  user-provided input. Only accept GitHub-hosted README URLs:
+  - `https://github.com/<owner>/<repo>/blob/<ref>/<path>`
+  - `https://github.com/<owner>/<repo>/raw/<ref>/<path>`
+  - `https://raw.githubusercontent.com/<owner>/<repo>/<ref>/<path>`
+
+  If the URL points anywhere else (or isn't a URL), **fail this check** and do not fetch it.
+- **Require the URL to point at a README file.** After stripping any fragment/query (see
+  below), the URL path must end with `README.md` (case-insensitive). If it points at some
+  other Markdown file, **fail this check** and ask the submitter to link the preset's README.
+- Fetch the **exact URL** in the `documentation` field. First strip any fragment (`#...`)
+  or query string (`?...`) — these are common when copying from the browser UI and must be
+  ignored so the fetch target is deterministic. Then resolve the raw content to fetch:
+  - For a `github.com/<owner>/<repo>/blob/<ref>/<path>` URL, fetch the equivalent
+    `github.com/<owner>/<repo>/raw/<ref>/<path>` URL (only swap `/blob/` → `/raw/`).
+  - Fetch `github.com/.../raw/...` and `raw.githubusercontent.com/...` URLs as-is.
+
+  Do **not** rewrite into `raw.githubusercontent.com/<owner>/<repo>/<ref>/<path>` form — that
+  format can't reliably represent refs containing slashes (e.g. a `feature/foo` branch).
+  Confirm the fetched URL resolves to a readable Markdown file.
+- **Validate that the README contains a valid Spec Kit CLI install command.** The fetched
+  README must contain at least one `specify preset add ...` invocation. The strongest
+  signal is the catalog-install form whose URL matches the submitted **Download URL**:
+  - `specify preset add --from <download-url>` (preferred), or
+  - `specify preset add <preset-id>`, or
+  - `specify preset add --dev <path>`
+
+  A `specify preset add --from <url>` command only counts when its `<url>` **matches the
+  submitted Download URL exactly**. A `--from` command pointing at a *different* URL does
+  **not** satisfy the install-command requirement (treat it as if absent) — but the README
+  may still pass on one of the other accepted forms (`specify preset add <preset-id>` or
+  `specify preset add --dev <path>`).
+
+  If **no** accepted `specify preset add ...` command is present, the README is treated as a
+  generic description/pitch rather than preset-usage documentation — **fail this check** and
+  tell the submitter to add a valid install command (ideally
+  `specify preset add --from <download-url>`).
+- **Prefer a preset-scoped README in monorepos.** If `documentation` resolves to a generic
+  repository-root README in a monorepo (the preset lives in a subdirectory such as
+  `presets/<id>/` and a preset-scoped README exists there), **flag it** in your comment and
+  recommend the submitter point `documentation` at the preset-scoped README
+  (e.g. `presets/<id>/README.md`) so the catalog surfaces usage instead of marketing. Treat
+  this as a flag rather than a hard failure **only if** the root README still contains a valid
+  `specify preset add ...` command for this preset; otherwise it fails check 2d above.
+
+### 2e. Release and download URL validation
 - The download URL should follow the pattern
  `https://github.com/<owner>/<repo>/archive/refs/tags/v<version>.zip`
  or
  `https://github.com/<owner>/<repo>/releases/download/<tag>/<asset>.zip`
 - Verify a GitHub release exists matching the submitted version

-### 2e. Submission checklists
+### 2f. Submission checklists
 - Confirm that all required checkboxes in the Testing Checklist and Submission
  Requirements sections are checked (`[x]`)

@@ -154,7 +208,7 @@ Insert the entry in **alphabetical order by preset ID** within the
    "repository": "<repository>",
    "download_url": "<download_url>",
    "homepage": "<homepage or repository>",
-    "documentation": "<documentation or repository README>",
+    "documentation": "<documentation URL — the validated preset-usage README>",
    "license": "<license>",
    "requires": {
      "speckit_version": "<speckit_version>"
--- a/presets/PUBLISHING.md
+++ b/presets/PUBLISHING.md
@@ -19,7 +19,7 @@ Before publishing a preset, ensure you have:

 1. **Valid Preset**: A working preset with a valid `preset.yml` manifest
 2. **Git Repository**: Preset hosted on GitHub (or other public git hosting)
-3. **Documentation**: README.md with description and usage instructions
+3. **Documentation**: A preset-scoped README.md that explains how to use **this preset**, including a valid `specify preset add ...` install command (see [Usage README Requirements](#usage-readme-requirements))
 4. **License**: Open source license file (MIT, Apache 2.0, etc.)
 5. **Versioning**: Semantic versioning (e.g., 1.0.0)
 6. **Testing**: Preset tested on real projects with `specify preset add --dev`
@@ -147,6 +147,46 @@ https://github.com/your-org/spec-kit-preset-your-preset/archive/refs/tags/v1.0.0
 specify preset add --from https://github.com/your-org/spec-kit-preset-your-preset/archive/refs/tags/v1.0.0.zip
 ```

+### Usage README Requirements
+
+The catalog `documentation` field must point at a README that explains how to use
+**this preset** — not a product pitch for a broader framework or a separate CLI.
+
+The submission workflow **mechanically enforces** that the linked README is a GitHub-hosted
+URL whose path ends with `README.md`, resolves to a readable file, and contains at least one
+valid `specify preset add ...` command. The remaining items (preferring a preset-scoped README
+in monorepos, covering the minimum structure) are expectations a human reviewer checks —
+follow them so your submission isn't sent back for changes.
+
+- **Point `documentation` at the preset-scoped README.** In a monorepo where the preset
+  lives in a subdirectory (e.g. `presets/<id>/`), link the README inside that directory
+  (`presets/<id>/README.md`) rather than the repository-root README. The root README is
+  often a marketing/overview page; the catalog should surface preset usage instead. The key
+  requirement is that this README is reachable at the `documentation` URL so users can read
+  it *before* downloading the release artifact — it's fine for the same file to also ship
+  inside the release ZIP.
+- **Include a valid Spec Kit CLI install command** *(enforced)*. The linked README must
+  contain at least one `specify preset add ...` invocation. Preferably use the
+  catalog-install form whose URL matches your Download URL:
+
+  ```bash
+  # <download-url> is the same URL you submit as the catalog Download URL —
+  # either the tag archive or a release asset, e.g.:
+  specify preset add --from https://github.com/<org>/<repo>/archive/refs/tags/vX.Y.Z.zip
+  specify preset add --from https://github.com/<org>/<repo>/releases/download/vX.Y.Z/<id>-X.Y.Z.zip
+  ```
+
+  `specify preset add <id>` and `specify preset add --dev <path>` are also accepted, but the
+  `--from <download-url>` form is the clearest signal that the README documents this exact
+  preset release.
+- **Cover the minimum structure** so a reader can decide whether the preset fits:
+  - What the preset does / what it provides
+  - The install command using Spec Kit CLI syntax (above)
+  - When to use it / when not to use it
+
+A submission whose linked README lacks a valid `specify preset add ...` command **fails
+validation** (workflow check 2d) and will not be added until corrected.
+
 ---

 ## Submit to Catalog
@@ -181,12 +221,14 @@ Edit `presets/catalog.community.json` and add your preset.
  "presets": {
    "your-preset": {
      "name": "Your Preset Name",
+      "id": "your-preset",
      "description": "Brief description of what your preset provides",
      "author": "Your Name",
      "version": "1.0.0",
      "download_url": "https://github.com/your-org/spec-kit-preset-your-preset/archive/refs/tags/v1.0.0.zip",
      "sha256": "OPTIONAL: SHA-256 hex digest of the archive above; verified before install",
      "repository": "https://github.com/your-org/spec-kit-preset-your-preset",
+      "documentation": "https://github.com/your-org/spec-kit-preset-your-preset/blob/main/README.md",
      "license": "MIT",
      "requires": {
        "speckit_version": ">=0.1.0"
@@ -243,7 +285,7 @@ git push origin add-your-preset

 ### Checklist
 - [ ] Valid preset.yml manifest
- [ ] README.md with description and usage
+- [ ] Usage README with a valid `specify preset add ...` command, linked from `documentation` (preset-scoped README recommended for monorepos)
 - [ ] LICENSE file included
 - [ ] GitHub release created
 - [ ] Preset tested with `specify preset add --dev`
@@ -264,7 +306,15 @@ After submission, maintainers will review:
 2. **Template quality** — templates are useful and well-structured
 3. **Command coherence** — commands reference sections that exist in templates
 4. **Security** — no malicious content, safe file operations
-5. **Documentation** — clear README explaining what the preset does
+5. **Documentation** — the README linked from `documentation` explains how to use *this* preset and contains a valid `specify preset add ...` command
+
+> **Reviewer note:** the workflow can mechanically check *structure* (the linked README
+> resolves and contains a valid `specify preset add ...` snippet; when that snippet uses the
+> `--from <url>` form, its URL must match the submitted download URL exactly — other accepted
+> forms like `specify preset add <id>` don't reference the download URL at all). Whether the
+> README genuinely documents *this* preset is partly a content judgment, so a human reviewer
+> should still confirm the linked doc isn't just a funnel to a separate product or CLI before
+> approving.

 Once verified, `verified: true` is set and the preset appears in `specify preset search`.