yzhao062/auditable
View on GitHub ↗Audit any agent decision across its past, present, and future, on one typed graph.
21 ★1 forksPythonUpdated 1mo ago
What you need to know
Open-source system of record for AI-agent decisions: it captures what each decision relied on, replays it against live state, and rolls back the committed action when it no longer holds — over one typed two-layer decision graph with PRE/LIVE/POST lifecycle pillars.
Install
pip install auditable pip install "auditable[graph]" pip install "auditable[langgraph]"
Usage
- •Wrap a LangGraph StateGraph with instrument() to get observed dependency edges with no change to node logic, then analyze_run(builder) to rank the run and name the keystone decision.
- •Lower a marked decision node to a DecisionRecord with builder.to_records(decisions={...}) and call replay(record, live_state=...) to re-decide under live state and route a rollback.
- •Lint a declared plan before deploy with analyze_plan (read-only structural lints: write_with_no_prior_read, flippable_dependency_annotation, scope_vs_snapshot, missing_revalidation_barrier).
- •Run the flagship end-to-end demo: pip install "auditable[graph]" && python examples/example_end_to_end.py.
Key features
- ✓One signed, hash-chained record binding three spans per decision (data, model, harness) with a prev_digest and content-addressed record_id.
- ✓Replay() returns ALLOW / ROLLBACK / BLOCK / HUMAN_REVIEW verdicts; ActionGate executes them through a compensating Rail that reverses committed actions.
- ✓Captures real runs via LangGraph instrument(), a LangChain callback, an in-place instrument at an MCP server boundary, or a framework-agnostic TouchRecorder.
- ✓POST analyze_run ranks finished runs by structural blast share; PRE analyze_plan withholds dependency-state risk (declared-only, never a false number).
- ✓Framework-agnostic, dependency-free, torch-free core with optional NetworkX (graph) and PyOD (anomaly) extras.
Best for
Reliability-conscious agent teams that need audit-and-recovery on top of existing tracing/evals/guardrails, especially for financially consequential agent actions.
Caveats
- ⚠The shipped ReferenceLedger rail is an in-process demo/test rail, not a production payment rail.
- ⚠POST structural score is an uncalibrated triage ranking; calibrated compound scoring, benchmark-grade detectors and automatic fixes are still on the roadmap.
- ⚠OWASP-Agentic/CWE rule floor for CI is planned, not shipping.
- ⚠CrewAI and OpenTelemetry integrations are roadmap items.
- ⚠Apache-2.0 licensed.
Platforms: LocalClients: Any MCP client (instrumented FastMCP server)
Reviewed 2026-08-18
Topics
agent-reliabilityagentic-aiai-agentsai-riskai-safetyanomaly-detectionlangchainllmllm-agentsllmopsmcpmodel-context-protocolobservabilitypythontrustworthy-ai
- Stars
- 21★
- Forks
- 1
- Language
- Python
- License
- Apache-2.0
- Created
- 2026-06-16
- Last push
- 2026-07-20