From bc6936159dddc4a192d2eea48216c107649fdbad Mon Sep 17 00:00:00 2001 From: Scrum4Me Agent <30029041+madhura68@users.noreply.github.com> Date: Wed, 6 May 2026 03:55:58 +0200 Subject: [PATCH] docs(ST-qfpqpxzy): worker-quota-probe.sh + uitgebreide pre-flight runbook + architectuurdoc - scripts/worker-quota-probe.sh: curl-probe die rate-limit-headers parset en { remaining, limit, pct, reset_at_iso } JSON retourneert - mcp-integration.md: pre-flight sectie uitgebreid met bash-script voorbeeld, retry-strategie (sleep tot reset+5s), bekende beperking - claude-question-channel.md: M13 quota-gate flow diagram + pre-flight loop uitleg Co-Authored-By: Claude Sonnet 4.6 --- docs/architecture/claude-question-channel.md | 22 ++++++++ docs/runbooks/mcp-integration.md | 56 +++++++++++--------- scripts/worker-quota-probe.sh | 44 +++++++++++++++ 3 files changed, 96 insertions(+), 26 deletions(-) create mode 100755 scripts/worker-quota-probe.sh diff --git a/docs/architecture/claude-question-channel.md b/docs/architecture/claude-question-channel.md index d4fc05a..dc7953c 100644 --- a/docs/architecture/claude-question-channel.md +++ b/docs/architecture/claude-question-channel.md @@ -75,5 +75,27 @@ weert). Dit patroon (notification-channel via een bestaande pg_notify-stream) is herbruikbaar — zie `docs/patterns/claude-question-channel.md`. +## Worker quota-gate flow (M13) + +De `worker_heartbeat` MCP-tool gebruikt hetzelfde `scrum4me_changes`-kanaal met een nieuw event-type `worker_heartbeat`: + +``` +Worker scrum4me-mcp Postgres SSE-route NavBar + | | | | | + |--worker_heartbeat(15,low)-->| | | | + | |--pg_notify('scrum4me_changes', {type:'worker_heartbeat',is_low:true})-->| + | | | |--data:{...}---->| + | | | | |--stand-by badge +``` + +**Pre-flight loop** (vóór elke `wait_for_job`): +1. `get_worker_settings` → `min_quota_pct` +2. `bash scripts/worker-quota-probe.sh` → `{ pct, reset_at_iso }` +3. `worker_heartbeat(last_quota_pct, last_quota_check_at, is_low = pct < min_quota_pct)` +4. Als `is_low`: log "wachten tot quota-reset om HH:MM", slaap tot `reset_at_iso + 5s`, herhaal stap 2 +5. Anders: `wait_for_job` aanroepen + +De solo-store houdt `lowQuotaTokenIds: Set` bij. De NavBar toont een stand-by badge wanneer `lowQuotaTokenIds.size >= connectedWorkers && connectedWorkers > 0`. + --- diff --git a/docs/runbooks/mcp-integration.md b/docs/runbooks/mcp-integration.md index 114e8f5..56c9fd9 100644 --- a/docs/runbooks/mcp-integration.md +++ b/docs/runbooks/mcp-integration.md @@ -79,46 +79,50 @@ Dit blijft gelden als je tussen jobs door commits, branches of pushes hebt gedaa ## Pre-flight quota-check -Vóór elke `wait_for_job` kan de worker controleren of er voldoende Claude-quota is. Stappenplan: +Vóór elke `wait_for_job` kan de worker controleren of er voldoende Claude-quota is. -1. Haal de drempel op: `get_worker_settings` → `min_quota_pct` (bijv. 20). -2. Probe de Claude API (rate-limit-headers): +**Helper-script:** `scripts/worker-quota-probe.sh` doet een minimale API-call (1 token) en geeft JSON terug: -```bash -RESPONSE=$(curl -sI https://api.anthropic.com/v1/messages \ - -H "x-api-key: $ANTHROPIC_API_KEY" \ - -H "anthropic-version: 2023-06-01") - -REMAINING=$(echo "$RESPONSE" | grep -i "anthropic-ratelimit-tokens-remaining" | awk '{print $2}' | tr -d '\r') -LIMIT=$(echo "$RESPONSE" | grep -i "anthropic-ratelimit-tokens-limit" | awk '{print $2}' | tr -d '\r') -RESET_EPOCH=$(echo "$RESPONSE" | grep -i "anthropic-ratelimit-tokens-reset" | awk '{print $2}' | tr -d '\r') - -QUOTA_PCT=$(( REMAINING * 100 / LIMIT )) +```json +{ "remaining": 180000, "limit": 200000, "pct": 90, "reset_at_iso": "2026-05-06T13:00:00Z" } ``` -3. Vergelijk met drempel. Bij low quota: rapporteer en slaap: +**Stappenplan:** + +1. Haal drempel op: `get_worker_settings` → `min_quota_pct` (bijv. 20). +2. Probe quota: ```bash -if [ "$QUOTA_PCT" -lt "$MIN_QUOTA_PCT" ]; then - RESET_MS=$(date -d "@$RESET_EPOCH" +%s%3N 2>/dev/null || echo 0) - NOW_MS=$(date +%s%3N) - SLEEP_S=$(( (RESET_MS - NOW_MS) / 1000 + 5 )) - echo "Wachten tot quota-reset om $(date -d "@$RESET_EPOCH" '+%H:%M')" - sleep "$SLEEP_S" -fi +PROBE=$(bash scripts/worker-quota-probe.sh) +PCT=$(echo "$PROBE" | grep -o '"pct":[0-9]*' | cut -d: -f2) +RESET_AT=$(echo "$PROBE" | grep -o '"reset_at_iso":"[^"]*"' | cut -d'"' -f4) ``` -4. Meld de quota-state via `worker_heartbeat`: +3. Vergelijk en meld via `worker_heartbeat`: ``` worker_heartbeat( - last_quota_pct=15, - last_quota_check_at="2026-05-06T12:00:00Z", - is_low=true + last_quota_pct=, + last_quota_check_at=, + is_low= ) ``` -5. Na de sleep: opnieuw controleren (stap 2) voordat `wait_for_job` wordt aangeroepen. +4. Bij low quota: retry-strategie — slaap tot reset + 5s: + +```bash +if [ "$PCT" -lt "$MIN_QUOTA_PCT" ]; then + RESET_S=$(date -d "$RESET_AT" +%s 2>/dev/null || date -j -f "%Y-%m-%dT%H:%M:%SZ" "$RESET_AT" +%s 2>/dev/null || echo 0) + NOW_S=$(date +%s) + SLEEP_S=$(( RESET_S - NOW_S + 5 )) + echo "Wachten tot quota-reset om $(date -d "$RESET_AT" '+%H:%M' 2>/dev/null || echo "$RESET_AT")" + [ "$SLEEP_S" -gt 0 ] && sleep "$SLEEP_S" +fi +``` + +5. Na de sleep: herhaal stap 2 (re-probe) voordat `wait_for_job` wordt aangeroepen. + +**Bekende beperking:** elke probe kost ~1 token; bij 12x/uur is de overhead 12 tokens/uur — verwaarloosbaar ten opzichte van taak-gebruik. ## Prompt diff --git a/scripts/worker-quota-probe.sh b/scripts/worker-quota-probe.sh new file mode 100755 index 0000000..662dde1 --- /dev/null +++ b/scripts/worker-quota-probe.sh @@ -0,0 +1,44 @@ +#!/usr/bin/env bash +# worker-quota-probe.sh — Probe Claude API rate-limit headers en output JSON. +# +# Vereist: ANTHROPIC_API_KEY env-var. +# Output: { "remaining": N, "limit": N, "pct": N, "reset_at_iso": "..." } +# Exit 0 bij succes, exit 1 bij fout (geen API key, curl-fout, parse-fout). + +set -euo pipefail + +if [ -z "${ANTHROPIC_API_KEY:-}" ]; then + echo '{"error":"ANTHROPIC_API_KEY not set"}' >&2 + exit 1 +fi + +HEADERS_FILE=$(mktemp /tmp/quota-probe-headers.XXXXXX) +trap 'rm -f "$HEADERS_FILE"' EXIT + +# Minimale API-aanroep (1 token) om rate-limit-headers te lezen. +HTTP_STATUS=$(curl -s -w "%{http_code}" -D "$HEADERS_FILE" -o /dev/null \ + -X POST https://api.anthropic.com/v1/messages \ + -H "x-api-key: $ANTHROPIC_API_KEY" \ + -H "anthropic-version: 2023-06-01" \ + -H "content-type: application/json" \ + -d '{"model":"claude-haiku-4-5-20251001","max_tokens":1,"messages":[{"role":"user","content":"."}]}') + +if [ "$HTTP_STATUS" -ge 500 ]; then + echo "{\"error\":\"API returned HTTP $HTTP_STATUS\"}" >&2 + exit 1 +fi + +# Parse headers (case-insensitive grep) +REMAINING=$(grep -i 'anthropic-ratelimit-tokens-remaining:' "$HEADERS_FILE" | awk '{print $2}' | tr -d '\r' || true) +LIMIT=$(grep -i 'anthropic-ratelimit-tokens-limit:' "$HEADERS_FILE" | awk '{print $2}' | tr -d '\r' || true) +RESET=$(grep -i 'anthropic-ratelimit-tokens-reset:' "$HEADERS_FILE" | awk '{print $2}' | tr -d '\r' || true) + +if [ -z "$REMAINING" ] || [ -z "$LIMIT" ] || [ "$LIMIT" -eq 0 ] 2>/dev/null; then + echo '{"error":"rate-limit headers not present in response"}' >&2 + exit 1 +fi + +PCT=$(( REMAINING * 100 / LIMIT )) + +printf '{"remaining":%s,"limit":%s,"pct":%s,"reset_at_iso":"%s"}\n' \ + "$REMAINING" "$LIMIT" "$PCT" "$RESET"