Case study · Systems & Infrastructure
DurableMCP
Durable execution for MCP tool calls
The problem
When an MCP client (Claude Desktop, Cursor, any agent) calls a tool, what happens if the executor crashes mid-call, the client retries, and the side effect may or may not have happened? DurableMCP makes that answer deterministic and inspectable.
What I built
Persist-before-dispatch
The server persists every call with a unique execution_id and idempotency_key before dispatching to an executor. Resubmitting the same (namespace, tool_name, idempotency_key) returns the original execution.
Fencing-token leases
Executors claim work with a monotonically increasing fencing token (execution_id, token, lease_expires, worker_id) and heartbeat while working. A stale executor resuming after lease expiry is rejected; the scheduler reaps expired leases and promotes retries.
Immutable event log
Every state transition is appended to an execution_events table — the schema is the proof, inspectable from Postgres through a read-only Next.js dashboard.
Real MCP protocol
JSON-RPC 2.0 / MCP 2025-03-26 over stdio and HTTP/SSE: initialize, tools/list, tools/call, ping. tools/call returns an execution_id; the executor runs work asynchronously.
Engineering decisions
At-least-once delivery, stated
Side-effecting tools supply their own idempotency key for external writes; the engine guarantees the state transition, not external idempotency.
Failure demos as scripts
fencing-demo.sh, duplicate-demo.sh and retry-demo.sh reproduce stale-worker rejection, duplicate submission, and retry exhaustion against the running stack — and the hosted demo kills its own executor to generate genuine crash-recovery events.
Raw pgx, no ORM
PostgreSQL repositories use raw pgx so the fencing and lease queries are explicit SQL a reviewer can audit.
Testing & CI
Compose stack boots Postgres (schema auto-applied), the MCP server, two executors (to demonstrate fencing), the scheduler and the dashboard; failure-scene scripts exercise the guarantees end to end.
Screenshots
Live executions dashboard
Execution totals, crash-rate chart, recovery events and the live executions table from the hosted demo
Go / MCP / PostgreSQL / Next.js