[ISS-3] Review dispatcher dedups on FAILED jobs: a failed PR review is never retried for the same head #268

Closed
opened 2026-09-28 03:52:26 +02:00 by janpeter · 0 comments
Owner

Beheerd door Scrum4Me — wijzigingen hier worden overschreven. Bron: https://thuis.jp-visser.nl/issues/cmukl9qxa002klj7rsxds8e8o

Status: closed (fixed) · Severity: s3_major · Gemeld door: scrum4me-server:claude · Occurrences: 1 (laatst: 2026-09-28T01:48:50.446Z) · PBI: PBI-195 · Aangemaakt: 2026-09-28T01:48:50.446Z

Registratie

lib/review-dispatch/enqueue.ts (enqueueReviewJob) checks for an existing job with findFirst({ where: { orchestration_key } }) regardless of status. The key is auto:pr:<owner>/<repo>#<n>@<head_sha>. A PR_REVIEW (or TASK_REVIEW) that ends FAILED blocks every later attempt for the same head: the cron (/api/cron/dispatch-reviews, every 5 min) reports it as skipped.dedup until someone pushes a new commit.

Observed (2026-09-28): agent-harness had 14 FAILED PR_REVIEW jobs in a row. 13 of them failed with post_pr_review_failed, a Forgejo 404 because the bot s4m-codex-reviewer was not a collaborator on the private repo. After the bot was added, PR #13 still did not get a new review. It only came through after the orchestration_key of the failed job was renamed by hand (…#failed-404-rerun); the new job cmukl52qz00c7vz7r5c9i6fhg was DONE/APPROVED.

Fix: dedup only on jobs that did not fail, for example where: { orchestration_key, status: { notIn: ['FAILED', 'CANCELLED'] } }, with a cap on the number of FAILED attempts per key (e.g. ≤ 3 within 24 h, counted via created_at). That way a permanent error (a 404 when access is missing) does not produce a new job every 5 minutes. Optionally a backoff based on the last finished_at. Test: FAILED job + the same candidate → enqueued; 3 FAILED → dedup/cap; QUEUED/CLAIMED/DONE → dedup (unchanged).

Related: the missing bot access was an onboarding gap (4 of 13 private repos). Consider a check in the dispatcher or health collector: private repo with a product but without the collaborator s4m-codex-reviewer → warning.

Onderzoek


2026-10-09T16:22:56.044Z — mac:claude

Fix in commit 9c9c8cef op branch fix/m49-issue-sweep (M49, T-2098), nog niet gepusht. enqueueReviewJob dedupt nu alleen op QUEUED/CLAIMED/RUNNING/DONE/SKIPPED; FAILED of CANCELLED laten een nieuwe poging toe, met een limiet van 3 per orchestration_key per 24 uur (uitkomst retry_limit, in de cron als skipped.retry_limit). Tests: RED 3/10 vóór de fix, daarna 33/33 groen. Praktijkbewijs op srv volgt na merge en deploy.

Oplossing


2026-10-09T18:05:18.810Z — mac:claude

Opgelost in M49: commit 9c9c8cef, PR #348 (merge e584e65c), uitgerold op scrum4me-server 2026-10-09 19:02 CEST via update_scrum4me_web (12/12 stappen ok).

Praktijkproef (2026-10-09): auto_pr_review tijdelijk aan voor my-mods, test-PR my-mods#18 (bot zonder repo-toegang, dus de review faalt bij het plaatsen). Resultaat voor sleutel auto:pr:janpeter/my-mods#18@2dbef7a3:

  • 19:05, 19:10, 19:15: drie PR_REVIEW-jobs, elk FAILED met post_pr_review_failed. Elke volgende cron-ronde plande de mislukte review opnieuw in (oude gedrag: na de eerste FAILED nooit meer).
  • 19:20–20:00: elke ronde skipped.retry_limit: 1, geen vierde job.
    Opgeruimd: auto_pr_review voor my-mods weer uit, PR #18 gesloten zonder merge, testbranch verwijderd.

Losse bevinding, buiten dit issue: s4m-codex-reviewer is geen collaborator op my-mods. Alleen relevant als auto-review daar weer aan gaat.

> Beheerd door Scrum4Me — wijzigingen hier worden overschreven. Bron: https://thuis.jp-visser.nl/issues/cmukl9qxa002klj7rsxds8e8o Status: closed (fixed) · Severity: s3_major · Gemeld door: scrum4me-server:claude · Occurrences: 1 (laatst: 2026-09-28T01:48:50.446Z) · PBI: PBI-195 · Aangemaakt: 2026-09-28T01:48:50.446Z ## Registratie `lib/review-dispatch/enqueue.ts` (`enqueueReviewJob`) checks for an existing job with `findFirst({ where: { orchestration_key } })` **regardless of status**. The key is `auto:pr:<owner>/<repo>#<n>@<head_sha>`. A PR_REVIEW (or TASK_REVIEW) that ends FAILED blocks every later attempt for the same head: the cron (`/api/cron/dispatch-reviews`, every 5 min) reports it as `skipped.dedup` until someone pushes a new commit. **Observed (2026-09-28):** agent-harness had 14 FAILED PR_REVIEW jobs in a row. 13 of them failed with `post_pr_review_failed`, a Forgejo 404 because the bot `s4m-codex-reviewer` was not a collaborator on the private repo. After the bot was added, PR #13 still did not get a new review. It only came through after the orchestration_key of the failed job was renamed by hand (`…#failed-404-rerun`); the new job cmukl52qz00c7vz7r5c9i6fhg was DONE/APPROVED. **Fix:** dedup only on jobs that did not fail, for example `where: { orchestration_key, status: { notIn: ['FAILED', 'CANCELLED'] } }`, with a cap on the number of FAILED attempts per key (e.g. ≤ 3 within 24 h, counted via `created_at`). That way a permanent error (a 404 when access is missing) does not produce a new job every 5 minutes. Optionally a backoff based on the last `finished_at`. Test: FAILED job + the same candidate → enqueued; 3 FAILED → dedup/cap; QUEUED/CLAIMED/DONE → dedup (unchanged). **Related:** the missing bot access was an onboarding gap (4 of 13 private repos). Consider a check in the dispatcher or health collector: private repo with a product but without the collaborator `s4m-codex-reviewer` → warning. ## Onderzoek --- *2026-10-09T16:22:56.044Z — mac:claude* Fix in commit 9c9c8cef op branch fix/m49-issue-sweep (M49, T-2098), nog niet gepusht. enqueueReviewJob dedupt nu alleen op QUEUED/CLAIMED/RUNNING/DONE/SKIPPED; FAILED of CANCELLED laten een nieuwe poging toe, met een limiet van 3 per orchestration_key per 24 uur (uitkomst `retry_limit`, in de cron als `skipped.retry_limit`). Tests: RED 3/10 vóór de fix, daarna 33/33 groen. Praktijkbewijs op srv volgt na merge en deploy. ## Oplossing --- *2026-10-09T18:05:18.810Z — mac:claude* Opgelost in M49: commit 9c9c8cef, PR #348 (merge e584e65c), uitgerold op scrum4me-server 2026-10-09 19:02 CEST via update_scrum4me_web (12/12 stappen ok). **Praktijkproef (2026-10-09):** auto_pr_review tijdelijk aan voor my-mods, test-PR my-mods#18 (bot zonder repo-toegang, dus de review faalt bij het plaatsen). Resultaat voor sleutel `auto:pr:janpeter/my-mods#18@2dbef7a3`: - 19:05, 19:10, 19:15: drie PR_REVIEW-jobs, elk FAILED met `post_pr_review_failed`. Elke volgende cron-ronde plande de mislukte review opnieuw in (oude gedrag: na de eerste FAILED nooit meer). - 19:20–20:00: elke ronde `skipped.retry_limit: 1`, geen vierde job. Opgeruimd: auto_pr_review voor my-mods weer uit, PR #18 gesloten zonder merge, testbranch verwijderd. Losse bevinding, buiten dit issue: s4m-codex-reviewer is geen collaborator op my-mods. Alleen relevant als auto-review daar weer aan gaat. <!-- s4m:issue:cmukl9qxa002klj7rsxds8e8o -->
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
janpeter/Scrum4Me#268
No description provided.