fix(symphony): correct runtime model — long-lived session, with multi-session support

Earlier framing assumed each /schedule-fired tick was a cold start
needing on-disk reconciliation. The actual model is a long-lived
orchestrator session where /schedule fires ticks within the running
session — TaskList sees prior-tick tasks just fine. Reframed
accordingly, while leaving the door open to other trigger sources
(Discord channel, webhook, second user session) which DO need
cross-session reconciliation.

Reconciliation logic now has a fast path and a slow path:

- Fast path: task in our TaskList → it's ours → reconcile via TaskGet.
- Slow path: task NOT in our TaskList (another session, or our session
  restarted) → check the worker's heartbeat file. Fresh → another
  session owns it, leave alone. Stale or missing → consider abandoned,
  retry; do not call TaskStop on a task we don't own.

Workers now write .symphony/heartbeats/<issue-id>.json on start,
refresh it at heartbeat_interval_ms cadence (default 60s), and delete
it on graceful exit. Stale heartbeats are how abandonment is detected
across sessions and after crashes.

Also added session_id to state.json running entries so cross-session
reconcilers know who spawned what.

WORKFLOW.md gains heartbeat_interval_ms and heartbeat_stale_ms knobs
under agent: with documented defaults.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
movq
2026-04-29 21:51:23 -05:00
parent 92f005895c
commit ee4e0b236e
5 changed files with 112 additions and 27 deletions

View File

@@ -5,7 +5,15 @@ description: Orchestrator skill for the Symphony plugin. Runs one tick — loads
# Symphony orchestrator (one tick)
You are the dispatcher. Each invocation runs **one** poll-and-dispatch pass and exits.
You are the dispatcher. Each invocation runs **one** poll-and-dispatch pass and returns.
The orchestrator runs in a long-lived session — `/schedule` fires ticks within the
running session rather than cold-starting. Tasks you spawn in tick N stay visible to
`TaskList` in tick N+1 *within the same session*.
Symphony also stays open to **other trigger sources** (Discord channel, webhook,
another user session). The reconciliation logic uses worker heartbeats to handle
those cases — see step 3.
Spec mapping is in `../../CLAUDE.md`.
## Inputs
@@ -41,7 +49,12 @@ their own.
```
.symphony/state.json
{
"running": { "<issue_id>": { "task_id": "...", "worktree": "...", "started_at": "..." } },
"running": { "<issue_id>": {
"task_id": "...",
"session_id": "...",
"worktree": "...",
"started_at": "..."
} },
"retry_attempts": { "<issue_id>": { "attempt": 2, "due_at_ms": 1714000000000, "last_error": "..." } },
"claimed": ["<issue_id>", ...],
"completed": ["<issue_id>", ...]
@@ -49,16 +62,27 @@ their own.
```
Create empty defaults if missing. Always rewrite atomically (write `state.json.tmp`, rename).
Session ID for entries we own is whatever this session reports; for entries owned by
other sessions, the field stays as the spawner wrote it.
### 3. Reconcile
For each entry in `running`:
- Call `TaskList` (or `TaskGet` by id). If the task is no longer running and reported success,
move issue to `completed`, drop from `running`.
- If the task failed, move into `retry_attempts` with exponential backoff (capped by
`agent.max_retry_backoff_ms`).
- Re-fetch the issue's current state from the tracker. If state is in `terminal_states` or
no longer in `active_states`, call `TaskStop` on the worker and drop from `running`.
**Fast path — task is in our `TaskList`** (we spawned it; same session):
- `TaskGet` for status. If completed successfully → move issue to `completed`, drop from `running`.
- If failed → move to `retry_attempts` with exponential backoff (capped by `agent.max_retry_backoff_ms`).
- Re-fetch issue state from the tracker. If in `terminal_states` or no longer in
`active_states` `TaskStop` and drop from `running`.
**Slow path — task is NOT in our `TaskList`** (spawned by another session, or our
session crashed and restarted):
- Read `.symphony/heartbeats/<issue-id>.json`.
- **Heartbeat fresh** (timestamp within `agent.heartbeat_stale_ms`, default 300000 = 5min)
→ another session is running it; do NOT dispatch a duplicate; leave alone.
- **Heartbeat stale OR missing** → consider abandoned. Move to `retry_attempts` (backoff
resets to attempt 1 if no prior retries; otherwise increments). Drop from `running`.
Delete the stale heartbeat file. Do NOT call `TaskStop` — we don't own the task.
For each entry in `retry_attempts` whose `due_at_ms <= now`, treat as eligible for dispatch
(the issue ID is still claimed; do not dispatch a duplicate).
@@ -101,7 +125,10 @@ For each candidate up to `slots`:
run_in_background: true
})
```
3. Record `running[issue.id] = { task_id, worktree, started_at: now }`.
3. Record `running[issue.id] = { task_id, session_id, worktree, started_at: now }`.
`session_id` should be a stable identifier for this orchestrator session — used by
other sessions to know who owns this task. If no session ID is available, omit the
field; reconcilers will fall back to heartbeat-only checks.
4. Drop from `retry_attempts` if present.
### 6. Persist & exit