Scrum4Me/docs/runbooks/review-plan-job.md

# Review-Plan Job Orchestration

> Implementation guide for the IDEA_REVIEW_PLAN job kind and multi-model iterative plan review.

---

## Overview

The review-plan job is an autonomous agent that performs iterative multi-model review of implementation plans (YAML frontmatter + markdown documents). It coordinates three review stages (structure, logic/patterns, risk assessment), detects convergence, and either approves the plan or returns it for manual refinement.

**Job Kind:** `IDEA_REVIEW_PLAN`
**Triggerable From:** `PLAN_READY`, `PLAN_REVIEWED` (re-review)
**Transitions To:** `PLAN_REVIEWED` (approved) or `PLAN_REVIEW_FAILED` (rejected/abandoned)

---

## System Design

### Data Flow

```
User clicks "Review Plan" on PLAN_READY idea
  ↓
startReviewPlanJobAction() queues IDEA_REVIEW_PLAN job
  ↓
Worker claims job via wait_for_job (MCP)
  ↓
Review-plan prompt orchestrates:
  - Ronde 1: Structure check (YAML parsing, format correctness)
  - Ronde 2: Logic & patterns (dependencies, architecture fit)
  - Ronde 3: Risk assessment (edge cases, refactoring, type-safety)
  ↓
Convergence detection: if stable, ask approval
  ↓
On approval: update_idea_plan_reviewed(approval_status='approved')
  → Idea transitions to PLAN_REVIEWED
  → IdeaLog entry created with PLAN_REVIEW_RESULT
  ↓
On rejection: return for manual edit (status → PLAN_REVIEW_FAILED)
```

### Review-Log JSON Schema

The orchestrator produces a detailed JSON log stored in `idea.plan_review_log`:

```typescript
interface ReviewLog {
  plan_file: string;           // Idea code (e.g., "I-042")
  created_at: ISO8601;         // Review start timestamp
  
  rounds: Array<{
    round: number;             // 0, 1, 2 (structure, logic, risk)
    model: string;             // claude-3-5-haiku | claude-3-5-sonnet | claude-opus-4-7
    role: string;              // "Structure Review" | "Logic & Patterns" | "Risk Assessment"
    focus: string;             // Review focus summary
    plan_before: string;       // Original plan_md at round start
    plan_after: string;        // Revised plan after feedback
    issues: Array<{
      category: 'structure' | 'logic' | 'risk' | 'pattern';
      severity: 'error' | 'warning' | 'info';
      suggestion: string;      // Concrete fix recommendation
    }>;
    score: number;             // 0-100 review score
    plan_diff_lines: number;   // Changed lines in this round
    converged: boolean;        // Did this round trigger convergence?
    timestamp: ISO8601;        // Round completion time
  }>;
  
  convergence?: {
    stable_at_round: number;   // Round where convergence was detected
    final_diff_pct: number;    // Percentage of changed lines at convergence
    convergence_metric: string; // "plan_stability" (constant for now)
  };
  
  approval: {
    status: 'pending' | 'approved' | 'rejected';
    timestamp?: ISO8601;       // When user made decision
  };
  
  summary: string;            // 1–2 sentence summary for IdeaLog
}
```

---

## Assumptions & Constraints

### Prompt Assumptions

1. **Plan Format:** Idea's `plan_md` field contains YAML frontmatter (parsed at PLAN_READY) + markdown body.
   - Frontmatter keys: `pbi`, `stories`, `tasks`, `priority`, `verify_required`.
   - If parse fails, orchestrator transitions idea to `PLAN_REVIEW_FAILED`.

2. **Context Availability:** The job payload includes:
   - `idea.plan_md`: The plan to review (required)
   - `idea.grill_md`: Context from grill phase (optional but recommended)
   - `product.definition_of_done`: Product-level acceptance criteria
   - `repo_url`: Local repository for pattern inspection

3. **User Availability:** At least one worker is active (server-side check via `countActiveWorkers`).

4. **No External APIs:** Orchestrator performs reviews entirely with information from job context. No external codex or multi-model APIs are called directly.
   - Future improvement: Codex-injection from `docs/patterns/**/*.md` and `docs/architecture/**/*.md`.

### Convergence Detection Assumptions

1. **Stability Metric:** Two consecutive rounds with < 5% line changes = convergence.
   - Threshold is hardcoded; future: make configurable per product.
   - Diff percentage = `(changed_lines / total_lines) * 100`.

2. **Max Iterations:** 3 initial rounds + 2 optional extra rounds (total max 5) before forced approval.

3. **No Infinite Loops:** If max iterations reached, approval gate enforces a decision.

### Validation Assumptions

1. **Plan is Mutable:** Orchestrator can revise `plan_md` between rounds without breaking downstream parsing.
   - If YAML structure is corrupted, `parsePlanMd` (server-side) will fail on approval.
   - Orchestrator should never corrupt YAML syntax.

2. **IdeaLog Persistence:** MCP tool `update_idea_plan_reviewed` atomically saves:
   - `idea.plan_review_log` (full JSON)
   - `idea.reviewed_at` (timestamp)
   - `idea.status` (transition)
   - `IdeaLog` entry (audit)

3. **User Decisions are Final:** Once approved, plan-review log is immutable (until next re-review).

---

## Implementation Details

### Prompt Location

- **Main Repo:** `lib/idea-prompts/review-plan-job.md`
- **MCP Server:** `scrum4me-mcp/src/prompts/idea/review-plan.md`
- **Synchronization:** Manual (for now); future: sync-schema.sh-like mechanism.

### Job Config Snapshot

Job created with config from `lib/job-config.ts`:

```typescript
IDEA_REVIEW_PLAN: {
  model: 'claude-opus-4-7',       // Opus for final orchestration
  thinking_budget: 6000,          // Extended for multi-round analysis
  permission_mode: 'acceptEdits',
  max_turns: 1,
  allowed_tools: [
    'Read', 'Write', 'Grep', 'Glob',
    'mcp__scrum4me__update_idea_plan_reviewed',
    'mcp__scrum4me__log_idea_decision',
    'mcp__scrum4me__update_job_status',
    'mcp__scrum4me__ask_user_question',
  ],
}
```

**Note:** Model is fixed to Opus for orchestration. Individual review rounds are simulated (not actual model switching) within Opus's analysis. Future: Direct multi-model support via Claude API.

### MCP Tool: update_idea_plan_reviewed

**Location:** `scrum4me-mcp/src/tools/update-idea-plan-reviewed.ts`

**Input:**
```typescript
{
  idea_id: string;
  review_log: object;  // Full ReviewLog JSON
  approval_status?: 'pending' | 'approved' | 'rejected';
}
```

**Behavior:**
1. Validates user owns idea.
2. Transitions idea status:
   - `approval_status='approved'` → `PLAN_REVIEWED`
   - `approval_status='rejected'` → `PLAN_REVIEW_FAILED`
   - Default → `PLAN_REVIEWED`
3. Saves `plan_review_log` and `reviewed_at` atomically.
4. Creates `IdeaLog` entry with type `PLAN_REVIEW_RESULT`.

---

## Dependencies

### Database

- **Idea Model:** Must have fields `plan_review_log` (Json), `reviewed_at` (DateTime).
- **IdeaStatus Enum:** Must include `REVIEWING_PLAN`, `PLAN_REVIEW_FAILED`, `PLAN_REVIEWED`.
- **IdeaLogType Enum:** Must include `PLAN_REVIEW_RESULT`.

### Server Actions

- `startReviewPlanJobAction()` — Queues job, enforces status transitions.
- `cancelIdeaJobAction()` — Allows user to cancel mid-review (reverts to `PLAN_READY`).

### MCP Tools

- `update_idea_plan_reviewed()` — Saves review-log and transitions status.
- `log_idea_decision()` — Logs convergence/approval decisions.
- `update_job_status()` — Marks job as done/failed.
- `ask_user_question()` — Approval gate interaction.

### Files

- `lib/idea-prompts/review-plan-job.md` — Orchestrator prompt.
- `scrum4me-mcp/src/prompts/idea/review-plan.md` — MCP server copy.
- `scrum4me-mcp/src/lib/kind-prompts.ts` — Prompt loader.
- `scrum4me-mcp/src/tools/wait-for-job.ts` — Job context builder.

---

## Error Handling

### Parse Failures

If `plan_md` cannot be parsed as valid YAML frontmatter:
1. Orchestrator logs error in review_log.
2. Calls `update_job_status('failed', error: 'plan_parse_failed')`.
3. Idea remains in `REVIEWING_PLAN` (no transition).
4. User can manually edit `plan_md` and retry.

### User Cancellation

If user cancels job via UI:
1. Server sets job status → `CANCELLED`.
2. Worker receives no further answer from `ask_user_question`.
3. Orchestrator gracefully saves partial review_log.
4. Calls `update_job_status('skipped', ...)`.
5. Idea reverts to `PLAN_READY`.

### Question Timeout

If approval question expires (24h):
1. Orchestrator logs timeout in review_log.
2. Calls `update_job_status('failed', error: 'approval_timeout')`.
3. Idea reverts to `PLAN_READY`.

---

## Testing Strategy

### Unit Tests

- **Mock ReviewLog Generation:** Verify review-log JSON structure matches schema.
- **Convergence Calculation:** Diff percentage computation, stability threshold.
- **Status Transitions:** Valid state machine paths (PLAN_READY → REVIEWING_PLAN → PLAN_REVIEWED).

### Integration Tests

- **End-to-End:** Draft idea → Grill → Plan → Review → PLAN_REVIEWED.
- **Re-Review:** PLAN_REVIEWED → REVIEWING_PLAN → PLAN_REVIEWED (no data loss).
- **Cancellation:** Mid-review cancellation → revert to PLAN_READY.
- **Parse Errors:** Malformed plan_md → PLAN_REVIEW_FAILED.

### Manual Testing

1. Create test idea with PLAN_READY status.
2. Click "Review Plan".
3. Monitor job in Jobs dashboard.
4. Verify review-log in idea detail page.
5. Accept/reject approval.
6. Confirm status transition and IdeaLog entry.

---

## Future Enhancements

1. **Direct Multi-Model Calls:** Use Claude API to invoke Haiku, Sonnet, Opus separately with model switching.
2. **Codex Injection:** Auto-load and inject `docs/patterns/**/*.md` and `docs/architecture/**/*.md` as context.
3. **Configurable Thresholds:** Allow product-level convergence percentage and max-rounds settings.
4. **Review History:** Preserve all review-logs for audit trail and re-review diffs.
5. **Feedback Loop:** Log user edits between review rounds and suggest re-run based on delta.
6. **Scheduled Re-Review:** Auto-trigger review after N days (staleness check).

---

## References

- `docs/architecture/jobs.md` — Job system architecture.
- `docs/patterns/server-action.md` — Server action pattern (startReviewPlanJobAction).
- `docs/api/rest-contract.md` — API surface for plan-review.
- `lib/idea-status.ts` — Status transition graph and state machine.
- `lib/idea-plan-parser.ts` — Plan YAML parsing (validator for approved plans).