Re-validate spec quality checklist after clarify updates spec (#2715)

* Re-validate spec quality checklist after clarify updates spec

After clarify modifies spec.md, the existing checklists/requirements.md
(generated by specify) can become stale. Items like 'No [NEEDS
CLARIFICATION] markers remain' may now pass, and newly added requirements
aren't reflected in the checklist evaluation.

Add step 8 to the clarify command that re-validates the spec quality
checklist against the updated spec after each clarification session:
- Check/uncheck items based on current spec state
- Report before/after pass counts in the completion report
- Skip silently if no checklist exists

Fixes #2693

* Address review: scope to checkbox lines, use FEATURE_DIR path

- Constrain re-validation to GitHub task-list checkbox lines only
  (- [ ] / - [x] outside code fences), ignoring headings, notes,
  and non-checkbox content
- Define pass counts as checked/total checkbox items
- Use FEATURE_DIR/checklists/requirements.md in Done When for
  consistency with the rest of the template

* Address review: handle regressions in checklist revalidation

- Clarify that each checkbox is set based solely on current spec state,
  regardless of prior marker (checked->unchecked is possible)
- Completion report now lists both newly passing items and regressions
  (checked->unchecked) so users see what became non-compliant

* Address review: case-insensitive checkboxes, preserve file verbatim

- Accept [x], [X], and leading whitespace for nested task items
- Explicitly state only the [ ]/[x] marker is toggled; all other
  file content (headings, metadata, notes, ordering, whitespace)
  must remain unchanged to avoid noisy diffs

* Address review: track per-item state, preserve marker case

- Add explicit before-snapshot step to capture each item's prior
  marker state before re-evaluation
- Compute three lists for the report: newly passing, regressions,
  and still unchecked
- Only toggle markers whose checked/unchecked state actually changes;
  preserve existing case ([x]/[X]) when state is unchanged to avoid
  cosmetic diffs
This commit is contained in:
Manfred Riem
2026-05-27 06:31:27 -05:00
committed by GitHub
parent 3cb7027fab
commit 9816f902ca

View File

@@ -197,6 +197,25 @@ Execution steps:
7. Write the updated spec back to `FEATURE_SPEC`.
8. **Re-validate Spec Quality Checklist** (if it exists):
- Check if `FEATURE_DIR/checklists/requirements.md` exists.
- If it does NOT exist, skip this step silently.
- If it exists:
1. Read the checklist file.
2. Identify all GitHub task-list checkbox lines — lines matching `- [ ]`, `- [x]`, or `- [X]` (case-insensitive, tolerant of leading whitespace for nested items) outside of code fences. Ignore all other content (headings, notes, non-checkbox bullets, metadata).
3. For each checkbox line, record its current marker state (checked or unchecked) and item text into a before-snapshot list.
4. Re-evaluate each checkbox item against the **updated** spec (the version just saved in step 7).
5. For each checkbox item, update only if the checked/unchecked state actually changes:
- If the item now passes and was unchecked: change `[ ]` to `[x]`.
- If the item now fails and was checked: change `[x]`/`[X]` to `[ ]`.
- If the state is unchanged: leave the marker as-is (preserve existing case to avoid cosmetic diffs).
6. Save the updated checklist file. **Only toggle the `[ ]`/`[x]` marker portion of checkbox lines whose state changed.** All other file content — headings, metadata, notes, line ordering, whitespace — must remain unchanged to avoid noisy diffs.
7. Compare the before-snapshot with the current state to compute three lists for the Completion Report:
- **Newly passing**: items that changed from unchecked to checked.
- **Regressions**: items that changed from checked to unchecked.
- **Still unchecked**: items that remain unchecked.
8. Record the before/after pass counts as checked/total checkbox items (e.g., "12/16 → 15/16 items passing").
Behavior rules:
- If no meaningful ambiguities found (or all potential questions would be low-impact), respond: "No critical ambiguities detected worth formal clarification." and suggest proceeding.
@@ -248,6 +267,7 @@ Report completion (after questioning loop ends or early termination):
- Number of questions asked & answered.
- Path to updated spec.
- Sections touched (list names).
- Spec quality checklist status (if `FEATURE_DIR/checklists/requirements.md` was re-validated): show before/after pass counts (e.g., "Spec Quality Checklist: 12/16 → 15/16 items passing") and list any items that changed state — both newly checked (unchecked → checked) and any regressions (checked → unchecked). If any items remain unchecked, list them as areas needing attention.
- Coverage summary table listing each taxonomy category with Status: Resolved (was Partial/Missing and addressed), Deferred (exceeds question quota or better suited for planning), Clear (already sufficient), Outstanding (still Partial/Missing but low impact).
- If any Outstanding or Deferred remain, recommend whether to proceed to `__SPECKIT_COMMAND_PLAN__` or run `__SPECKIT_COMMAND_CLARIFY__` again later post-plan.
- Suggested next command.
@@ -255,5 +275,6 @@ Report completion (after questioning loop ends or early termination):
## Done When
- [ ] Spec ambiguities identified and clarifications integrated into spec file
- [ ] Spec quality checklist re-validated against updated spec (if `FEATURE_DIR/checklists/requirements.md` exists)
- [ ] Extension hooks dispatched or skipped according to the rules in Mandatory Post-Execution Hooks above
- [ ] Completion reported to user with questions answered, sections touched, and coverage summary
- [ ] Completion reported to user with questions answered, sections touched, checklist status, and coverage summary