fix(symphony): correct runtime model — long-lived session, with multi-session support
Earlier framing assumed each /schedule-fired tick was a cold start needing on-disk reconciliation. The actual model is a long-lived orchestrator session where /schedule fires ticks within the running session — TaskList sees prior-tick tasks just fine. Reframed accordingly, while leaving the door open to other trigger sources (Discord channel, webhook, second user session) which DO need cross-session reconciliation. Reconciliation logic now has a fast path and a slow path: - Fast path: task in our TaskList → it's ours → reconcile via TaskGet. - Slow path: task NOT in our TaskList (another session, or our session restarted) → check the worker's heartbeat file. Fresh → another session owns it, leave alone. Stale or missing → consider abandoned, retry; do not call TaskStop on a task we don't own. Workers now write .symphony/heartbeats/<issue-id>.json on start, refresh it at heartbeat_interval_ms cadence (default 60s), and delete it on graceful exit. Stale heartbeats are how abandonment is detected across sessions and after crashes. Also added session_id to state.json running entries so cross-session reconcilers know who spawned what. WORKFLOW.md gains heartbeat_interval_ms and heartbeat_stale_ms knobs under agent: with documented defaults. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -5,7 +5,15 @@ description: Orchestrator skill for the Symphony plugin. Runs one tick — loads
|
||||
|
||||
# Symphony orchestrator (one tick)
|
||||
|
||||
You are the dispatcher. Each invocation runs **one** poll-and-dispatch pass and exits.
|
||||
You are the dispatcher. Each invocation runs **one** poll-and-dispatch pass and returns.
|
||||
The orchestrator runs in a long-lived session — `/schedule` fires ticks within the
|
||||
running session rather than cold-starting. Tasks you spawn in tick N stay visible to
|
||||
`TaskList` in tick N+1 *within the same session*.
|
||||
|
||||
Symphony also stays open to **other trigger sources** (Discord channel, webhook,
|
||||
another user session). The reconciliation logic uses worker heartbeats to handle
|
||||
those cases — see step 3.
|
||||
|
||||
Spec mapping is in `../../CLAUDE.md`.
|
||||
|
||||
## Inputs
|
||||
@@ -41,7 +49,12 @@ their own.
|
||||
```
|
||||
.symphony/state.json
|
||||
{
|
||||
"running": { "<issue_id>": { "task_id": "...", "worktree": "...", "started_at": "..." } },
|
||||
"running": { "<issue_id>": {
|
||||
"task_id": "...",
|
||||
"session_id": "...",
|
||||
"worktree": "...",
|
||||
"started_at": "..."
|
||||
} },
|
||||
"retry_attempts": { "<issue_id>": { "attempt": 2, "due_at_ms": 1714000000000, "last_error": "..." } },
|
||||
"claimed": ["<issue_id>", ...],
|
||||
"completed": ["<issue_id>", ...]
|
||||
@@ -49,16 +62,27 @@ their own.
|
||||
```
|
||||
|
||||
Create empty defaults if missing. Always rewrite atomically (write `state.json.tmp`, rename).
|
||||
Session ID for entries we own is whatever this session reports; for entries owned by
|
||||
other sessions, the field stays as the spawner wrote it.
|
||||
|
||||
### 3. Reconcile
|
||||
|
||||
For each entry in `running`:
|
||||
- Call `TaskList` (or `TaskGet` by id). If the task is no longer running and reported success,
|
||||
move issue to `completed`, drop from `running`.
|
||||
- If the task failed, move into `retry_attempts` with exponential backoff (capped by
|
||||
`agent.max_retry_backoff_ms`).
|
||||
- Re-fetch the issue's current state from the tracker. If state is in `terminal_states` or
|
||||
no longer in `active_states`, call `TaskStop` on the worker and drop from `running`.
|
||||
|
||||
**Fast path — task is in our `TaskList`** (we spawned it; same session):
|
||||
- `TaskGet` for status. If completed successfully → move issue to `completed`, drop from `running`.
|
||||
- If failed → move to `retry_attempts` with exponential backoff (capped by `agent.max_retry_backoff_ms`).
|
||||
- Re-fetch issue state from the tracker. If in `terminal_states` or no longer in
|
||||
`active_states` → `TaskStop` and drop from `running`.
|
||||
|
||||
**Slow path — task is NOT in our `TaskList`** (spawned by another session, or our
|
||||
session crashed and restarted):
|
||||
- Read `.symphony/heartbeats/<issue-id>.json`.
|
||||
- **Heartbeat fresh** (timestamp within `agent.heartbeat_stale_ms`, default 300000 = 5min)
|
||||
→ another session is running it; do NOT dispatch a duplicate; leave alone.
|
||||
- **Heartbeat stale OR missing** → consider abandoned. Move to `retry_attempts` (backoff
|
||||
resets to attempt 1 if no prior retries; otherwise increments). Drop from `running`.
|
||||
Delete the stale heartbeat file. Do NOT call `TaskStop` — we don't own the task.
|
||||
|
||||
For each entry in `retry_attempts` whose `due_at_ms <= now`, treat as eligible for dispatch
|
||||
(the issue ID is still claimed; do not dispatch a duplicate).
|
||||
@@ -101,7 +125,10 @@ For each candidate up to `slots`:
|
||||
run_in_background: true
|
||||
})
|
||||
```
|
||||
3. Record `running[issue.id] = { task_id, worktree, started_at: now }`.
|
||||
3. Record `running[issue.id] = { task_id, session_id, worktree, started_at: now }`.
|
||||
`session_id` should be a stable identifier for this orchestrator session — used by
|
||||
other sessions to know who owns this task. If no session ID is available, omit the
|
||||
field; reconcilers will fall back to heartbeat-only checks.
|
||||
4. Drop from `retry_attempts` if present.
|
||||
|
||||
### 6. Persist & exit
|
||||
|
||||
Reference in New Issue
Block a user