mirror of
https://github.com/github/spec-kit.git
synced 2026-08-03 06:26:30 +08:00
208d38695f
* feat(extensions): add assess idea assessment pipeline extension Add a role-neutral, opt-in "Idea Assessment Pipeline" extension (id: assess) covering the discovery work that happens BEFORE spec-driven development. It provides a five-stage funnel: intake, research, define, shape, decide, each writing one artifact under .specify/assessments/<slug>/. A go verdict hands off to /speckit.specify; killing an idea is a first-class success outcome. Registration: - extensions/catalog.json: bundled core opt-in entry (before bug) - pyproject.toml: force-include maps into core_pack so it ships in the installed wheel (verified via wheel build) Also normalizes a Rich-wrapped substring assertion in test_workflows.py so the suite passes at CI's 80-column non-TTY width. Assisted-by: GitHub Copilot (model: Claude Opus 4.8, supervised) Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 1f25cf9d-b7eb-4b2b-b811-3e91d8db8f6a * fix(extensions): address PR review on assess extension Resolve review feedback on github/spec-kit#3568: - catalog.json: bump top-level updated_at to this revision (2026-07-17) - extension.yml + catalog.json: shorten the assess description to under the documented 200-char manifest limit (kept aligned across both) - extension.yml: make the before_specify hook prompt condition-neutral (it fires on every /speckit.specify, so it must not claim "no assessment found") - intake.md: fix slug normalization to explicitly allow lowercase letters a-z (the old rule permitted only digits and '-', contradicting the offline-mode example) - intake.md + research.md: require a sanitized source URL (strip userinfo and credential/signature query params) instead of persisting a verbatim URL that could leak secrets into project artifacts - decide.md: remove the "trivially small" exception so a go always requires a shaped concept, making verdict behavior deterministic and consistent with the guardrails and README Assisted-by: GitHub Copilot (model: Claude Opus 4.8, supervised) Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 1f25cf9d-b7eb-4b2b-b811-3e91d8db8f6a * refactor(extensions): remove before_specify hook from assess Assess is a separate business process from spec-driven development, so it should not inject itself into the /speckit.specify lifecycle. The hook fired on every /speckit.specify invocation (it had no condition), nagging even when an assessment already existed and the user was deliberately proceeding. Unlike git's before_specify (a mechanical prerequisite: create a feature branch) or agent-context's after_* hooks (reacting to spec output), assess is an upstream, optional, human-judgment process. The coupling that belongs here already runs forward and by choice: a `go` verdict from /speckit.assess.decide hands off to /speckit.specify. The backward hook was the redundant, intrusive direction. - extension.yml: drop the hooks block (commands-only manifest) - README.md: replace the Hooks section with a Handoff section - test: replace the hook assertion with test_declares_no_hooks to lock in the standalone-pipeline design Assisted-by: GitHub Copilot (model: Claude Opus 4.8, supervised) Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 1f25cf9d-b7eb-4b2b-b811-3e91d8db8f6a * fix(extensions): harden assess slug handling and clarify verdict logic Address the second review round on github/spec-kit#3568: - Slug path traversal: intake and all four downstream commands (research, define, shape, decide) now normalize an explicit or user-supplied slug to the [a-z0-9-] alphabet (dropping '.', '/', '\\') and reject an empty normalized result before constructing ASSESS_DIR. This guarantees a slug like `../..` cannot escape .specify/assessments/. - Metadata accuracy: the extension.yml and catalog.json descriptions no longer imply a "build/kill" call is handed to /speckit.specify — only a `go` hands off; a `kill` closes the assessment. - Verdict determinism (decide): a `go` now explicitly requires evidence strength `adequate`+ (never weak/unknown), resolving the conflict with the thin-evidence guardrail. - Risk polarity (decide): renamed the "Risk" criterion to "Risk posture" with positive polarity (strong = risks understood and mitigated) so it composes with the other scores that feed the verdict. - README: aligned the go-threshold guardrail with the evidence rule and documented the slug-normalization safety property. The PR description was also updated to drop the stale before_specify hook claim (the hook was removed in the previous commit). Assisted-by: GitHub Copilot (model: Claude Opus 4.8, supervised) Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 1f25cf9d-b7eb-4b2b-b811-3e91d8db8f6a * fix(extensions): add symlink/realpath containment and pin research host allowlist Address the third review round on github/spec-kit#3568: - Path safety (intake, research, define, shape, decide): slug normalization blocks lexical `..` but not symlinked path components. Each command now, before any mkdir/read/write, resolves the real path of .specify/assessments/<slug>/ and every artifact, refuses to follow a symlinked .specify / assessments / slug dir / artifact, and verifies the resolved path stays inside the project root. This blocks a cloned or crafted project from redirecting reads/writes outside the repository. Each stage enforces this independently since research/define/decide can run without intake. - research URL policy: replaced the open-ended "and comparable well-known hosts" no-prompt branch with intake's exact enumerated allowlist, so an agent cannot classify an attacker-controlled host as "comparable" and fetch it without confirmation. - README: guardrail now documents symlink/realpath containment. Assisted-by: GitHub Copilot (model: Claude Opus 4.8, supervised) Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 1f25cf9d-b7eb-4b2b-b811-3e91d8db8f6a * fix(extensions): redact secrets in captured idea and stop on explicit-slug collision Address the fourth review round on github/spec-kit#3568 (intake): - Secret leak in the captured idea: quoting the original "verbatim" contradicted the URL sanitization rule when the idea itself contained a credential-bearing URL. Capture now redacts secrets (sanitize URLs; strip tokens, passwords, keys, cookies) inside the quoted text as well as the Source field, and the section heading is "Idea (as captured)" rather than "verbatim". - Explicit-slug collision: in automated mode an existing intake.md caused a silent switch to a new slug, contradicting the no-suffix guarantee for user-provided slugs. Now: user-provided slug collision -> stop and report; only a self-generated slug (already disambiguated at resolution) is re-slugged. Assisted-by: GitHub Copilot (model: Claude Opus 4.8, supervised) Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 1f25cf9d-b7eb-4b2b-b811-3e91d8db8f6a * fix(extensions): reject IPv6 private ranges and DNS-rebinding in URL policy Address the remaining open comment from review 4722852090 on github/spec-kit#3568 (the other six comments in that round were already resolved by the slug-validation and host-allowlist fixes in9cd07fbandc032a2e). The URL Trust Policy refused only textual IPv4 loopback/RFC1918/metadata hosts, so an approved hostname resolving to an internal IPv6 or IPv4-mapped address could still reach internal services. The refuse- outright list now covers IPv6 link-local (fe80::/10), unique-local (fc00::/7), IPv4-mapped forms, and the IPv6 metadata address, and adds a resolution-time check: even an allowlisted or user-confirmed host is refused when it resolves to any non-public address, defeating DNS rebinding. Mirrored the summary in research's inherited-policy note. Assisted-by: GitHub Copilot (model: Claude Opus 4.8, supervised) Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 1f25cf9d-b7eb-4b2b-b811-3e91d8db8f6a * fix(extensions): pin connection vs DNS rebinding and gate slug-only direct entry Address the fifth review round on github/spec-kit#3568: - DNS rebinding (intake + research): a standalone DNS lookup does not defeat rebinding because the fetch client can re-resolve or pick a private address from a mixed answer. The policy now requires the fetch to pin the connection to a validated public address (or verify the connected peer) and re-apply the refusal ranges to the address actually connected to; if the fetch mechanism cannot pin or expose the peer, the fetch is refused rather than trusted by hostname. - Slug-only direct entry (research + define): when intake/research artifacts are absent and $ARGUMENTS carries only a slug, the commands no longer infer an idea/problem from the slug. They now require substantive idea/problem text and otherwise prompt (interactive) or stop (automated). - Cleaned up a leftover duplicate ASSESS_DIR assignment line in research. Assisted-by: GitHub Copilot (model: Claude Opus 4.8, supervised) Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 1f25cf9d-b7eb-4b2b-b811-3e91d8db8f6a * fix(extensions): allow read-only source inspection in intake guardrail Address the sixth review round on github/spec-kit#3568. The intake guardrail said the command "only reads and writes inside .specify/assessments/<slug>/", which contradicts its documented inputs: intake must read a codebase pointer (repository inspection) and fetch an allowed URL to capture the idea. The guardrail now limits only *writes* to the assessment directory and explicitly permits read-only inspection of the supplied sources (repo + allowlisted URL fetch under the URL Trust Policy). The other four commands already phrased this correctly ("read only, and write inside ...") and are unchanged. Assisted-by: GitHub Copilot (model: Claude Opus 4.8, supervised) Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 1f25cf9d-b7eb-4b2b-b811-3e91d8db8f6a * Harden assess commands: ancestor path safety, untrusted-artifact reads, research dir creation Addresses review 4723905370 on PR #3568 across three themes: - Ancestor path safety: verify `.specify` and `.specify/assessments` are real directories (not symlinks) resolving inside the project root before any filesystem-based slug resolution, in all five commands. - Untrusted artifact reads: treat the contents of persisted assessment artifacts (intake/research/problem/concept) as untrusted data, not instructions — ignore embedded directives, mirroring the URL Trust Policy. - research now ensures the validated ASSESS_DIR exists before writing, since it may be the first assessment command run. Assisted-by: GitHub Copilot (model: Claude Opus 4.8, supervised) Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 1f25cf9d-b7eb-4b2b-b811-3e91d8db8f6a * Allow absent assessment dir in ancestor path-safety check Addresses review 4723955260 on PR #3568. The ancestor path-safety clause required `.specify/assessments` to already be a real directory, which blocked the first-run commands (intake, research, define) from ever reaching the step that creates it. Reword the clause in all five commands so a not-yet-created directory is permitted, while still refusing when `.specify` or `.specify/assessments` exists as a symlink or escapes the project root. Assisted-by: GitHub Copilot (model: Claude Opus 4.8, supervised) Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 1f25cf9d-b7eb-4b2b-b811-3e91d8db8f6a * Fix README diagram: needs-clarification revisits the named earlier stage Addresses review 4724027270 on PR #3568. The overview flowchart routed every needs-clarification verdict back to research, but decide.md's Revisit stage can send an idea back to intake, research, define, or shape. Reroute the arrow as a generic loop back to the earlier stages so the diagram no longer misstates the pipeline. Assisted-by: GitHub Copilot (model: Claude Opus 4.8, supervised) Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 1f25cf9d-b7eb-4b2b-b811-3e91d8db8f6a * Make decide handoff integration-neutral (no hard-coded dot-style) Addresses review 4724080304 on PR #3568: - decide.md frontmatter description hard-coded `/speckit.specify`. Frontmatter is parsed before command-reference resolution, so it now uses agent-neutral wording ("hand survivors off into Spec-Driven Development") instead of a dot-style literal that would be wrong for non-dot integrations. - The `## If go — Handoff to …` heading inside the decision.md output template hard-coded `/speckit.specify`, which would be written verbatim into decision.md. It now uses the `__SPECKIT_COMMAND_SPECIFY__` placeholder, like the rest of the command, so the active integration's invocation style is rendered. Assisted-by: GitHub Copilot (model: Claude Opus 4.8, supervised) Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 1f25cf9d-b7eb-4b2b-b811-3e91d8db8f6a --------- Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
104 lines
6.1 KiB
Markdown
104 lines
6.1 KiB
Markdown
# Idea Assessment Pipeline Extension
|
||
|
||
A five-stage assessment pipeline for Spec Kit that turns **any idea** into a defensible **go / needs-clarification / kill** decision *before* it enters Spec-Driven Development. It is the missing **discovery track** that sits in front of the SDD **delivery track** (`specify → clarify → plan → tasks → analyze → implement`).
|
||
|
||
Discovery answers *"is this worth building?"* Delivery answers *"how do we build it?"* Only ideas that survive assessment hand off to `/speckit.specify`.
|
||
|
||
## Overview
|
||
|
||
Each idea lives in its own directory under `.specify/assessments/<slug>/`, with one Markdown artifact per stage:
|
||
|
||
```
|
||
.specify/assessments/<slug>/
|
||
├── intake.md # speckit.assess.intake — capture the raw idea
|
||
├── research.md # speckit.assess.research — gather (and challenge with) evidence
|
||
├── problem.md # speckit.assess.define — define the problem, goals, metrics
|
||
├── concept.md # speckit.assess.shape — shape solution options + appetite
|
||
└── decision.md # speckit.assess.decide — go / needs-clarification / kill → handoff
|
||
```
|
||
|
||
The pipeline is a **funnel**: most ideas should be killed or parked before `shape`. Killing an idea with a documented reason is a successful outcome, not a failure.
|
||
|
||
```mermaid
|
||
flowchart LR
|
||
A[intake] --> R[research] --> D[define] --> S[shape] --> C{decide}
|
||
C -->|go| SPEC[/speckit.specify/]
|
||
C -->|kill| X[closed, recorded]
|
||
C -.->|needs-clarification: revisit the named earlier stage| A
|
||
```
|
||
|
||
## Commands
|
||
|
||
| Command | Stage | Output |
|
||
|---------|-------|--------|
|
||
| `speckit.assess.intake` | Capture & normalize a raw idea (text, URL, ticket, or codebase pointer). | `intake.md` |
|
||
| `speckit.assess.research` | Gather users/market/prior-art/data evidence — and evidence *against* the idea. | `research.md` |
|
||
| `speckit.assess.define` | Define the problem: users, goals, non-goals, success metrics, cost of inaction. | `problem.md` |
|
||
| `speckit.assess.shape` | Shape 2–3 concept-level options with appetite and trade-offs; recommend one (or none). | `concept.md` |
|
||
| `speckit.assess.decide` | Score against criteria and render the verdict; hand `go` ideas to `/speckit.specify`. | `decision.md` |
|
||
|
||
Stages are meant to run in order but are not rigidly gated:
|
||
|
||
- `define` is the minimum viable stage and can run directly on user input (intake/research optional).
|
||
- `shape` requires `problem.md`.
|
||
- `decide` requires `problem.md`; a `go` verdict expects `concept.md` (otherwise it is downgraded to `needs-clarification`).
|
||
|
||
## Slug Conventions
|
||
|
||
A *slug* is the per-idea directory name under `.specify/assessments/`. It is the handle all five commands share.
|
||
|
||
- **User-provided**: normalized to lowercase kebab-case (e.g. `offline-mode`, `cut-onboarding-friction`). Preserved verbatim after normalization — no timestamps or numbers appended.
|
||
- **Asked for**: in interactive use, `speckit.assess.intake` asks for a slug when none is supplied, suggesting a kebab-case default derived from the idea.
|
||
- **Automated**: when no human is available, the agent generates a unique slug and never overwrites an existing assessment directory (appending `-2`, `-3`, … or a short date as needed).
|
||
- **Reuse from context**: later stages reuse the slug reported earlier in the same session, confirmed by the presence of the assessment directory.
|
||
|
||
## Installation
|
||
|
||
```bash
|
||
specify extension add assess
|
||
```
|
||
|
||
## Disabling
|
||
|
||
```bash
|
||
specify extension disable assess
|
||
specify extension enable assess
|
||
```
|
||
|
||
## Typical Flow
|
||
|
||
```bash
|
||
# 1. Capture an idea (pasted text, a URL, or "assess this repo")
|
||
/speckit.assess.intake "Let users work offline and sync when they reconnect" slug=offline-mode
|
||
|
||
# 2. Gather evidence — and reasons it might not be worth it
|
||
/speckit.assess.research slug=offline-mode
|
||
|
||
# 3. Define the actual problem
|
||
/speckit.assess.define slug=offline-mode
|
||
|
||
# 4. Shape 2–3 concept options with appetites
|
||
/speckit.assess.shape slug=offline-mode
|
||
|
||
# 5. Decide — go, clarify, or kill
|
||
/speckit.assess.decide slug=offline-mode
|
||
# → on "go", hand the decision.md handoff summary to /speckit.specify
|
||
```
|
||
|
||
## Handoff
|
||
|
||
`assess` is a **standalone pipeline you enter deliberately** — it registers no lifecycle hooks and never inserts itself into `/speckit.specify`. The only coupling runs forward and by choice: a `go` verdict from `/speckit.assess.decide` hands its `decision.md` summary to `/speckit.specify`. Discovery and specification stay separate processes.
|
||
|
||
## Guardrails
|
||
|
||
- Only `speckit.assess.*` commands write, and only inside `.specify/assessments/<slug>/`. **None of them modify source code** — solution design and implementation belong to the SDD lifecycle (`/speckit.specify` onward).
|
||
- Web content fetched during `intake`/`research` is treated as untrusted data, governed by an explicit URL Trust Policy (allowlisted public sources fetched freely; unknown hosts prompted or skipped; loopback/RFC1918/metadata endpoints refused).
|
||
- Evidence is never over-claimed: unsourced statements are tagged `ASSUMPTION`, and `research.md` always includes an *Evidence Against the Idea* section.
|
||
- Verdicts are never over-claimed: a `go` requires a valid problem, `adequate`+ evidence (never weak/unknown), and a shaped concept; otherwise the honest verdict is `needs-clarification`.
|
||
- Slugs are normalized to `[a-z0-9-]` and an empty result is rejected; before any read or write, each command also rejects symlinked path components and verifies the resolved path stays inside the project root — so an assessment can never escape `.specify/assessments/`, even in a crafted or cloned project.
|
||
- No command overwrites an existing artifact without confirmation; in automated mode it refuses.
|
||
|
||
## Relationship to Other Extensions
|
||
|
||
`assess` is deliberately the **generic, role-neutral** discovery track — usable by a founder, PM, BA, engineer, or designer. Richer or more specialized pre-SDD flows in the community catalog (e.g. product-lifecycle orchestrators, technical-discovery, intake-normalization, brownfield onboarding) can layer on top of or feed into it; `assess` aims to be the minimal, opinionated funnel that ends cleanly at the `/speckit.specify` handoff.
|