AgentFork
A counterfactual debugger for black-box LLM agents in stateful environments. Tracing tells you what an agent did; AgentFork tells you what would have happened otherwise. Fork a recorded trajectory at any step, rebuild the Postgres world exactly as it was, change one thing (the model, what retrieval returned, a tool's reply, what memory held), then run it forward N times to get an outcome distribution instead of a guess. Agents are recorded as OpenTelemetry GenAI spans with content-addressed payloads at 0.0037% overhead. Replays fan out over a Redis Streams queue with leases, reclaim on worker death, idempotent results and a shared token-bucket rate limiter, each in its own isolated database across five isolation strategies that are ablated for leaks by state hashing. On injected faults with no oracle access, adding replay-based intervention to trace analysis raised layer-attribution accuracy from 79.7% to 96.9% (McNemar p < 0.001), and a regression gate replays past incidents to block bad config changes.
- Python
- PostgreSQL
- Redis Streams
- OpenTelemetry
- MCP
- LLM agents
- Causal inference
