9.4 KiB
9.4 KiB
Question Set — Plan Interviewer
42 questions across 5 sections. Read this file at the start of every interview.
Lineage tags: [SK] Spec Kit · [K] Kiro/EARS · [OS] OpenSpec · [BM] BMAD · [ADR] Nygard/MADR · [PRFAQ] Amazon Working Backwards · [RFC] Rust RFC · [PM] Pre-mortem (Klein) · [SDD] SDD papers · [CC] Claude Code guidance · [SOLO] Solo-dev priority
Spec — 12 questions
Problem & motivation
- In one sentence: what hurts today that this fixes? [PRFAQ]
- What's the current workaround? There is always a current workaround. [PRFAQ]
- Why now? What changed that makes this worth building this week and not six months ago? [RFC]
Scope
- List three things this project will deliberately NOT do. [SK gap-fill]
- Is there anything in your draft that's a "maybe later" that should be moved to an explicit "NOT NOW" list? [SK]
User stories & primary journey
- Walk me through each primary user journey, step by step, in present tense, as if I were the user. [SK + PRFAQ]
Acceptance criteria (applied per criterion)
- For each acceptance criterion: rewrite it in EARS form —
WHEN [trigger] THE SYSTEM SHALL [response]. If you can't, leave as prose and flag it. Soft requirement. [EARS] - For each user story, name the single observable behavior you could check in a browser/CLI to know it works. Soft requirement — may bundle with #7's answer when natural. [SK Independent Test]
Edge cases
- What happens on: empty state? Network/dependency failure? Concurrent edit? Action triggered twice quickly? Largest/smallest input outside the expected range? [K + SK consolidated]
Success criteria
- For each adjective in your spec — "fast", "simple", "responsive", "secure", "intuitive", "robust" — give me a number and a unit, or strike the adjective. [SK ambiguity pass]
- What's the success metric for this project as a whole? If you can't tell whether it succeeded six weeks from now, your metric is wrong. [SK + PRFAQ]
Hygiene scan (closes section)
- Are there any
[NEEDS CLARIFICATION], TODO, ???,<placeholder>markers left? Read the spec out loud — does it mention any language, library, framework, file path, API endpoint, or database? Those are implementation leaks, move to Plan. [SK verbatim]
Red flags during Spec:
- "should" instead of MUST/SHALL
- Vague adjectives without thresholds
- Acceptance criteria written as test steps
- Tech stack in the spec
- No explicit non-goals
- Contradictory user stories
Plan — 17 questions
Decision interrogation (applied per non-trivial decision — hybrid in-line ADR loop)
- What is the decision? State it as: "We will use X." [ADR]
- What did you reject, and why? Highest-leverage Plan question. [ADR/MADR]
- Is this a one-way door (hard to reverse — Postgres-vs-Mongo) or a two-way door (easy — UI library)? [Bezos framing]
- What does this make easier? What does it make harder? [ADR — Nygard verbatim]
Data model
- List every entity. For each: fields, types, relationships, uniqueness constraints, indexes you anticipate. [SK data-model]
- What's the migration path from current state to this schema? What's the rollback story if it turns out wrong? [OS brownfield-first]
Interfaces / contracts
- For each external interface (HTTP route, CLI command, MCP tool, library function): inputs, outputs, error shapes including status codes and retry semantics. [SK contracts + K]
- What's the versioning story? What happens to existing callers when this changes? [RFC backwards-compat]
Deployment
- Where does this run, and how does it get from your editor to that environment? What secrets does it need and where are they stored? [SOLO consolidated]
- What's the smallest reversible deploy unit? Can you ship Story 1 without Story 2? [SK independent-test]
Tests
- What's the testing posture for this project: TDD, test-after, or skip? State it explicitly — it changes task ordering. [SK + CC]
- For each user story, name the test that would fail today and pass when it's done. [CC test-first][SK Independent Test]
NFRs
- What's the latency budget, throughput target, or concurrency limit if any of those matter for this project? (If none matter, say so explicitly.) [SK measurability] — easy to skip with n/a
- Threat model in one paragraph: what's the attacker, what are they after, what's the trust boundary? What gets logged and where does the log go? [SDD + observability] — easy to skip with n/a
Over-engineering scan
- Is anything in this plan over-engineered for the scope? AI tends to over-build — flag anything that smells gold-plated. [SK verbatim warning][SOLO]
Phasing
- What's Phase 0 — the irreducible foundation everything else needs? What can be built in parallel vs. truly depends on what? [SK plan-template + SK [P]]
- What's the first PR that delivers any user-visible value? [SOLO]
End-of-Plan safety pass (always run)
After completing the Plan section, run a single safety pass:
"Decisions I caught in this session: [list each one with its rejected alternatives]. Any I missed?"
This catches decisions the in-line ADR loop missed without forcing a front-loaded enumeration.
Red flags during Plan:
- Decision without rationale (no rejected alternatives)
- Consequences section only lists positives
- Schema in spec instead of plan
- No testing strategy stated
- No deploy/rollback story
- Contradiction with constitution (auto-CRITICAL)
Tasks — 4 questions
- Where are the natural decomposition points in this work? Those are your phases. [phasing]
- Which phases require something only you can do — provision infrastructure, provide a secret, run a deploy from privileged hardware? Flag them so the agent knows where to pause and what's needed. [human-required blockers]
- Walk the Spec's requirement list (FR-001, FR-002, ...). For each requirement, which phase satisfies it? Any orphans either direction — requirements without a phase, or phases without requirements? [exhaustive coverage check]
- Are any phases iteration-shaped rather than build-shaped? Flag them so the agent treats them as converge-with-Travis loops rather than finish-and-handoff. Done is "Travis is happy", not a spec criterion. [iterate-vs-build — Travis-specific]
Notes:
- Tasks is the section most likely to be skipped on small projects. If the work is small enough that one ordered list of TODOs is overkill, skip the whole section with one acknowledgment.
- Question 3 requires the Spec requirements to be numbered (FR-001 etc). The skill should number them as it captures them in Spec.
Constitution — 4 questions
Visit this section twice: once briefly at the open, once as a cross-check at the close of the whole interview.
Open (Q1)
- Does this project have a constitution — a short list of principles that apply across every feature, not just this one? If yes, state them. If no, is that intentional (small project) or should we draft a few before continuing?
Per-principle interrogation (Q2-3, looped per principle if any exist)
- State this principle in one sentence containing MUST or MUST NOT. Give me one concrete decision it would have changed in past work — if you can't, it's too vague to be a principle. [SK + BM]
- How will the agent know mid-implementation whether it has violated this? Automatable check (lint/test/CI) or human-detectable only? [SDD Constitutional SDD]
Close — cross-check (Q4, at end of whole interview)
- Walk through each principle: does any element of the Spec, Plan, or Tasks violate it? Constitution violations are critical — fix the plan, not the principle. [SK constitution-alignment pass — auto-CRITICAL]
Notes:
- Most small projects don't need a constitution. Accept "no constitution, too small" and skip the section.
- Be ready for the answer to #3 sometimes being "no automatable check, just my judgment." That's fine — but knowing it's only judgment is itself useful.
Cross-cutting — 5 questions (closes the interview)
Codebase assumptions
- What in the existing codebase is this plan assuming exists and works? What is it relying on but should not touch? Flag both — assumptions are the most common source of broken implementations. [OS brownfield + SOLO]
Unresolved questions
- What's still unresolved? List it explicitly — must-resolve-before-code vs. can-defer-and-tag-in-code vs. out-of-scope. [RFC verbatim distinction]
Pre-mortem
- Imagine it is 6 weeks from now. This project failed badly — not "ran late," actively went wrong. What happened? [PM — Klein verbatim framing]
- Is there anything in this plan you secretly suspect is wrong but haven't surfaced because surfacing it would mean more work? Surface it now. [PM candor — Klein verbatim "things you wouldn't ordinarily mention"]
Go/no-go (ends the interview)
- Would you hand this plan to Lovebug right now and walk away? Yes / not yet / no. If not yet or no — what's the one or two things missing?
Notes:
- The pre-mortem (Q3-4) is the single highest-value question pair in the whole interview. Klein's research shows prospective hindsight produces 30% better failure identification, and the mechanism works for individuals, not just teams.
- The "secretly suspect is wrong" framing matters. Don't soften it to "any concerns?" — the wording does work.
Total: 42 questions
Typical session asks 15–25 depending on what the user arrived with and how much is skipped. The 42 is the ceiling, not a target.