From 9816f902ca1bcec2e3a261b03baad75eb3109aba Mon Sep 17 00:00:00 2001 From: Manfred Riem <15701806+mnriem@users.noreply.github.com> Date: Wed, 27 May 2026 06:31:27 -0500 Subject: [PATCH] Re-validate spec quality checklist after clarify updates spec (#2715) * Re-validate spec quality checklist after clarify updates spec After clarify modifies spec.md, the existing checklists/requirements.md (generated by specify) can become stale. Items like 'No [NEEDS CLARIFICATION] markers remain' may now pass, and newly added requirements aren't reflected in the checklist evaluation. Add step 8 to the clarify command that re-validates the spec quality checklist against the updated spec after each clarification session: - Check/uncheck items based on current spec state - Report before/after pass counts in the completion report - Skip silently if no checklist exists Fixes #2693 * Address review: scope to checkbox lines, use FEATURE_DIR path - Constrain re-validation to GitHub task-list checkbox lines only (- [ ] / - [x] outside code fences), ignoring headings, notes, and non-checkbox content - Define pass counts as checked/total checkbox items - Use FEATURE_DIR/checklists/requirements.md in Done When for consistency with the rest of the template * Address review: handle regressions in checklist revalidation - Clarify that each checkbox is set based solely on current spec state, regardless of prior marker (checked->unchecked is possible) - Completion report now lists both newly passing items and regressions (checked->unchecked) so users see what became non-compliant * Address review: case-insensitive checkboxes, preserve file verbatim - Accept [x], [X], and leading whitespace for nested task items - Explicitly state only the [ ]/[x] marker is toggled; all other file content (headings, metadata, notes, ordering, whitespace) must remain unchanged to avoid noisy diffs * Address review: track per-item state, preserve marker case - Add explicit before-snapshot step to capture each item's prior marker state before re-evaluation - Compute three lists for the report: newly passing, regressions, and still unchecked - Only toggle markers whose checked/unchecked state actually changes; preserve existing case ([x]/[X]) when state is unchanged to avoid cosmetic diffs --- templates/commands/clarify.md | 23 ++++++++++++++++++++++- 1 file changed, 22 insertions(+), 1 deletion(-) diff --git a/templates/commands/clarify.md b/templates/commands/clarify.md index d4eb0048a..8488557e7 100644 --- a/templates/commands/clarify.md +++ b/templates/commands/clarify.md @@ -197,6 +197,25 @@ Execution steps: 7. Write the updated spec back to `FEATURE_SPEC`. +8. **Re-validate Spec Quality Checklist** (if it exists): + - Check if `FEATURE_DIR/checklists/requirements.md` exists. + - If it does NOT exist, skip this step silently. + - If it exists: + 1. Read the checklist file. + 2. Identify all GitHub task-list checkbox lines — lines matching `- [ ]`, `- [x]`, or `- [X]` (case-insensitive, tolerant of leading whitespace for nested items) outside of code fences. Ignore all other content (headings, notes, non-checkbox bullets, metadata). + 3. For each checkbox line, record its current marker state (checked or unchecked) and item text into a before-snapshot list. + 4. Re-evaluate each checkbox item against the **updated** spec (the version just saved in step 7). + 5. For each checkbox item, update only if the checked/unchecked state actually changes: + - If the item now passes and was unchecked: change `[ ]` to `[x]`. + - If the item now fails and was checked: change `[x]`/`[X]` to `[ ]`. + - If the state is unchanged: leave the marker as-is (preserve existing case to avoid cosmetic diffs). + 6. Save the updated checklist file. **Only toggle the `[ ]`/`[x]` marker portion of checkbox lines whose state changed.** All other file content — headings, metadata, notes, line ordering, whitespace — must remain unchanged to avoid noisy diffs. + 7. Compare the before-snapshot with the current state to compute three lists for the Completion Report: + - **Newly passing**: items that changed from unchecked to checked. + - **Regressions**: items that changed from checked to unchecked. + - **Still unchecked**: items that remain unchecked. + 8. Record the before/after pass counts as checked/total checkbox items (e.g., "12/16 → 15/16 items passing"). + Behavior rules: - If no meaningful ambiguities found (or all potential questions would be low-impact), respond: "No critical ambiguities detected worth formal clarification." and suggest proceeding. @@ -248,6 +267,7 @@ Report completion (after questioning loop ends or early termination): - Number of questions asked & answered. - Path to updated spec. - Sections touched (list names). +- Spec quality checklist status (if `FEATURE_DIR/checklists/requirements.md` was re-validated): show before/after pass counts (e.g., "Spec Quality Checklist: 12/16 → 15/16 items passing") and list any items that changed state — both newly checked (unchecked → checked) and any regressions (checked → unchecked). If any items remain unchecked, list them as areas needing attention. - Coverage summary table listing each taxonomy category with Status: Resolved (was Partial/Missing and addressed), Deferred (exceeds question quota or better suited for planning), Clear (already sufficient), Outstanding (still Partial/Missing but low impact). - If any Outstanding or Deferred remain, recommend whether to proceed to `__SPECKIT_COMMAND_PLAN__` or run `__SPECKIT_COMMAND_CLARIFY__` again later post-plan. - Suggested next command. @@ -255,5 +275,6 @@ Report completion (after questioning loop ends or early termination): ## Done When - [ ] Spec ambiguities identified and clarifications integrated into spec file +- [ ] Spec quality checklist re-validated against updated spec (if `FEATURE_DIR/checklists/requirements.md` exists) - [ ] Extension hooks dispatched or skipped according to the rules in Mandatory Post-Execution Hooks above -- [ ] Completion reported to user with questions answered, sections touched, and coverage summary +- [ ] Completion reported to user with questions answered, sections touched, checklist status, and coverage summary