yzhao062/auditable

View on GitHub ↗

Audit any agent decision across its past, present, and future, on one typed graph.

21 ★1 forksPythonUpdated 1mo ago

What you need to know

Open-source system of record for AI-agent decisions: it captures what each decision relied on, replays it against live state, and rolls back the committed action when it no longer holds — over one typed two-layer decision graph with PRE/LIVE/POST lifecycle pillars.

Install

pip install auditable
pip install "auditable[graph]"
pip install "auditable[langgraph]"

Usage

  • Wrap a LangGraph StateGraph with instrument() to get observed dependency edges with no change to node logic, then analyze_run(builder) to rank the run and name the keystone decision.
  • Lower a marked decision node to a DecisionRecord with builder.to_records(decisions={...}) and call replay(record, live_state=...) to re-decide under live state and route a rollback.
  • Lint a declared plan before deploy with analyze_plan (read-only structural lints: write_with_no_prior_read, flippable_dependency_annotation, scope_vs_snapshot, missing_revalidation_barrier).
  • Run the flagship end-to-end demo: pip install "auditable[graph]" && python examples/example_end_to_end.py.

Key features

  • One signed, hash-chained record binding three spans per decision (data, model, harness) with a prev_digest and content-addressed record_id.
  • Replay() returns ALLOW / ROLLBACK / BLOCK / HUMAN_REVIEW verdicts; ActionGate executes them through a compensating Rail that reverses committed actions.
  • Captures real runs via LangGraph instrument(), a LangChain callback, an in-place instrument at an MCP server boundary, or a framework-agnostic TouchRecorder.
  • POST analyze_run ranks finished runs by structural blast share; PRE analyze_plan withholds dependency-state risk (declared-only, never a false number).
  • Framework-agnostic, dependency-free, torch-free core with optional NetworkX (graph) and PyOD (anomaly) extras.

Best for

Reliability-conscious agent teams that need audit-and-recovery on top of existing tracing/evals/guardrails, especially for financially consequential agent actions.

Caveats

  • The shipped ReferenceLedger rail is an in-process demo/test rail, not a production payment rail.
  • POST structural score is an uncalibrated triage ranking; calibrated compound scoring, benchmark-grade detectors and automatic fixes are still on the roadmap.
  • OWASP-Agentic/CWE rule floor for CI is planned, not shipping.
  • CrewAI and OpenTelemetry integrations are roadmap items.
  • Apache-2.0 licensed.
Platforms: LocalClients: Any MCP client (instrumented FastMCP server)

Reviewed 2026-08-18

Topics

agent-reliabilityagentic-aiai-agentsai-riskai-safetyanomaly-detectionlangchainllmllm-agentsllmopsmcpmodel-context-protocolobservabilitypythontrustworthy-ai
Stars
21★
Forks
1
Language
Python
License
Apache-2.0
Created
2026-06-16
Last push
2026-07-20