fix(symphony): correct runtime model — long-lived session, with multi-session support
Earlier framing assumed each /schedule-fired tick was a cold start needing on-disk reconciliation. The actual model is a long-lived orchestrator session where /schedule fires ticks within the running session — TaskList sees prior-tick tasks just fine. Reframed accordingly, while leaving the door open to other trigger sources (Discord channel, webhook, second user session) which DO need cross-session reconciliation. Reconciliation logic now has a fast path and a slow path: - Fast path: task in our TaskList → it's ours → reconcile via TaskGet. - Slow path: task NOT in our TaskList (another session, or our session restarted) → check the worker's heartbeat file. Fresh → another session owns it, leave alone. Stale or missing → consider abandoned, retry; do not call TaskStop on a task we don't own. Workers now write .symphony/heartbeats/<issue-id>.json on start, refresh it at heartbeat_interval_ms cadence (default 60s), and delete it on graceful exit. Stale heartbeats are how abandonment is detected across sessions and after crashes. Also added session_id to state.json running entries so cross-session reconcilers know who spawned what. WORKFLOW.md gains heartbeat_interval_ms and heartbeat_stale_ms knobs under agent: with documented defaults. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
movq
@@ -29,9 +29,12 @@ repo to provide the integration. See "Tracker contract" below.
|
||||
## User flow
|
||||
|
||||
1. **First time**: `/symphony-init` in the project root. Interview + write.
|
||||
2. **Steady state**: `/symphony-tick` is fired by a `/schedule` cron entry (also set up
|
||||
by init unless the user declines).
|
||||
3. **Re-config**: re-run `/symphony-init`; it detects drift and proposes updates.
|
||||
2. **Steady state**: `/symphony-tick` is fired by a `/schedule` entry inside the long-lived
|
||||
orchestrator session (also set up by init unless the user declines).
|
||||
3. **Other triggers**: ticks can also be fired by remote channels (Discord DM, webhook,
|
||||
another session). The reconciliation logic handles cross-session liveness via worker
|
||||
heartbeats.
|
||||
4. **Re-config**: re-run `/symphony-init`; it detects drift and proposes updates.
|
||||
|
||||
## Tracker contract
|
||||
|
||||
@@ -66,31 +69,65 @@ The skill and worker read these instructions when they need to talk to the track
|
||||
the project CLAUDE.md is silent on the tracker, the orchestrator errors with
|
||||
`tracker_integration_missing` and exits the tick.
|
||||
|
||||
## State
|
||||
## Runtime model
|
||||
|
||||
Symphony's spec calls for in-memory orchestrator state. Cron-fired ticks are cold starts,
|
||||
so we persist to `.symphony/state.json` in the consuming repo:
|
||||
The orchestrator runs inside a **long-lived Claude Code session**. `/schedule` doesn't
|
||||
cold-start each tick — it sets the prompt for the running session. Tasks spawned in
|
||||
tick N stay visible to `TaskList` / `TaskGet` in tick N+1 within that session, so the
|
||||
session itself is the primary store of "what's running right now."
|
||||
|
||||
- `running` — issue_id → { worker_task_id, started_at, worktree_path }
|
||||
Symphony stays open to **other trigger sources** too: a Discord channel could DM the
|
||||
session and ask for a tick, a webhook hook could trigger one, and a separate user
|
||||
session could conceivably spawn tasks against the same project. To keep all three
|
||||
modes honest, two layers carry state:
|
||||
|
||||
1. **`TaskList` / `TaskGet`** — fast path, authoritative for liveness *within this session*.
|
||||
2. **`.symphony/state.json` + worker heartbeats** — durable record across sessions and
|
||||
restarts. Authoritative when `TaskList` doesn't know about a task ID.
|
||||
|
||||
### Reconciliation logic
|
||||
|
||||
For each entry in `state.json`'s `running` map:
|
||||
|
||||
- **In our `TaskList`** → it's our task; check status via `TaskGet`; reconcile normally.
|
||||
- **Not in our `TaskList`** → another session may own it, or the session that spawned it
|
||||
exited. Read the worker's heartbeat file (`.symphony/heartbeats/<issue-id>.json`).
|
||||
- **Heartbeat fresh** (within `agent.heartbeat_stale_ms`, default 5min) → leave alone;
|
||||
another session is owning it.
|
||||
- **Heartbeat stale** → consider abandoned; mark for retry, drop from `running`.
|
||||
- **No heartbeat** → ditto.
|
||||
|
||||
This keeps the single-session case simple (TaskList wins) while supporting
|
||||
multi-session and crash recovery cleanly.
|
||||
|
||||
### State file shape
|
||||
|
||||
- `running` — issue_id → { task_id, session_id, started_at, worktree_path }
|
||||
- `retry_attempts` — issue_id → { attempt, due_at_ms, last_error }
|
||||
- `claimed` — set of issue IDs reserved this tick (cleared on dispatch or release)
|
||||
- `completed` — set of issue IDs (bookkeeping)
|
||||
|
||||
Reconciliation reads the file at the top of every tick, queries `TaskList` to confirm
|
||||
which workers are actually still running, and rewrites the file before returning.
|
||||
### Worker heartbeats
|
||||
|
||||
Workers write `.symphony/heartbeats/<issue-id>.json` with `{ timestamp, status, task_id, session_id }`
|
||||
every minute (configurable via `agent.heartbeat_interval_ms`). On graceful exit they
|
||||
remove the file. Stale heartbeats — and stale `running` entries with no heartbeat at
|
||||
all — are how Symphony detects abandonment.
|
||||
|
||||
## Mapping to the spec
|
||||
|
||||
| Symphony component | Claude Code mechanism |
|
||||
|---------------------------|--------------------------------------------------------|
|
||||
| Polling daemon | `/schedule` cron entry firing `/symphony-tick` |
|
||||
| Polling daemon | `/schedule` firing `/symphony-tick` in a long-lived session |
|
||||
| Workflow loader | Skill body — reads & parses `WORKFLOW.md` |
|
||||
| Issue tracker client | Project CLAUDE.md instructions + whatever tools fit |
|
||||
| Orchestrator | Skill, invoked once per tick; state on disk |
|
||||
| Orchestrator | Skill, runs each tick within the same session |
|
||||
| Workspace manager | `EnterWorktree` per worker |
|
||||
| Agent runner | `Agent` tool with `subagent_type: symphony-worker` |
|
||||
| Hooks | Bash steps inside the worker prompt |
|
||||
| Concurrency cap | Lockfile + `TaskList` poll inside the dispatcher |
|
||||
| Concurrency cap | `TaskList` count + `state.json` cross-check |
|
||||
| Retries with backoff | `retry_attempts` entries, due_at compared each tick |
|
||||
| Liveness across sessions | Worker heartbeats in `.symphony/heartbeats/` |
|
||||
| Logging | `.symphony/logs/<issue>-<ts>.log` |
|
||||
|
||||
## Known divergences from the spec
|
||||
|
||||
Reference in New Issue
Block a user