IBM/AssetOpsBench

AssetOpsBench - Industry 4.0: A unified benchmark and framework for building, orchestrating, and evaluating domain-specific AI agents for Industry 4.0 asset operations and maintenance, with 460+ scenarios, 5 specialist agents (IoT, FMSR, TSFM, Work Order,...), and multi-agent orchestration blueprints (MetaAgent, AgentHive) over MCP.

2,087 ★296 forksPythonUpdated 5d ago

What you need to know

Open research framework (benchmark + agent tooling) for building, orchestrating, and evaluating domain-specific AI agents in industrial asset operations and maintenance (Industry 4.0), with native MCP support via domain-specific MCP servers (IoT, FMSR, TSFM, Work Order, Vibration) and 141+ reproducible scenarios.

Install

git clone https://github.com/IBM/AssetOpsBench.git && cd AssetOpsBench && pip install -e .
Or run instantly on Colab / the HuggingFace Playground

Usage

  • •python -m assetopsbench.run --scenario "List all sensors of Chiller 6 in MAIN site"
  • •Follow INSTRUCTIONS.md for full setup, MCP servers, and the plan-execute runner
  • •Choose an agent framework: Plan Execute, Deep Agent, Claude Agent, or OpenAI Agent

Key features

  • ✓9 asset classes, 141+ scenarios, 5 domain agents across IoT, FMSR, time-series (TSFM), work orders, and vibration
  • ✓Domain-specific MCP servers; the TSFM server exposes 41 tools (model catalog, feature catalog, recipes, evaluation, run/results)
  • ✓Multiple agent frameworks including plan-and-execute and ReAct-based orchestrators with agent-as-tool delegation
  • ✓Leaderboards: 7 LLMs scored by an LLM judge (Llama-4-Maverick-17B) on 6-dimensional criteria
  • ✓Powers public AI agent competitions (IJCAI 2026 live, CODS 2025 completed with 500+ submissions)

Caveats

  • ⚠This is a research/benchmark framework, not a standalone production MCP server
  • ⚠Requires LLM + time-series foundation model API access (model access enabled via TokenRouter)
  • ⚠Active development is on main; per-publication code lives on separate branches
Platforms: Linux/macOS (Python) · Colab
Documentation ↗

Reviewed 2026-08-07

Topics

ai-for-physical-assetscondition-based-maintenancehvac-maintenanceiotllm-agentsmodel-context-protocolpredictive-maintenancetime-series-forecasting
View on GitHub ↗
Stars
2,087★
Forks
296
Language
Python
License
Apache-2.0
Created
2025-05-01
Last push
2026-08-02