Case study · Product & AI
SupportPilot AI
Supervised AI support operations with human-approved write actions
The problem
AI support tooling fails in two directions: ungrounded answers that invent policy, and agents that take financial actions nobody approved. SupportPilot treats the model as an untrusted drafter inside a supervised ops system with deterministic evaluation.
What I built
Traced agent pipeline
Every ticket runs injection scan → 13-intent classification with calibrated confidence → context retrieval (lookup_order, check_inventory, search_policy) → grounded drafting → action proposal → confidence scoring.
Grounded drafting with a post-check
The model must cite evidence inline as [POL:REF-101] / [ORD:ORD-21001]; a post-check extracts every amount, date, duration and code in the draft and flags anything not traceable to evidence.
Human approval gate
WRITE-tier tools (refund_preview, cancel_order_preview) are blocked from auto-run and queued as proposals only. Approving computes a pure-math preview and records an audit event; executing is admin-only and confirm-gated. No funds ever move.
Policy retrieval
Tagged policy chunks searched by pgvector cosine similarity with a lexical overlap boost, transparently falling back to deterministic in-process hashing embeddings when the extension or column is unavailable.
RBAC capability map
Server-side enforcement: agent can draft/reject, senior_agent can approve, admin can execute and run evals. The approval state machine rejects double decisions (409) and refuses execution before approval.
Engineering decisions
Injection containment by construction
Untrusted customer text is pattern-scanned and delimited; flagged tickets run in safe mode with zero write proposals, turning prompt injection from a data-integrity risk into a routing decision.
Evaluation before vibes
36 seeded cases across 13 intents plus 3 prompt-injection fixtures, scored by deterministic graders (intent match, actions correct, no auto-execution, citations present, all claims grounded, injection contained) with an LLM rubric judge grading tone separately.
No financial rails by design
ALLOW_REAL_EXECUTION=false: execution is recorded as executed_simulated. The demo proves the supervision model without needing real money to move.
Testing & CI
Deterministic eval harness (task success, wrong-action rate, unsupported claim rate, cost per case, p50/p95 latency, tone) reproducible offline with the mock provider; runs persist to eval_runs. Postgres 17 + pgvector.
Screenshots
Ticket workspace
Grounded draft with inline evidence citations beside the customer thread
Approval gate
Write actions queued as proposals; approval computes a pure-math preview and records an audit event
Injection safe mode
Flagged tickets run with write tools disabled and SEC-901 cited
Next.js / PostgreSQL / pgvector / Gemini