vortico/flama

pip install flama
pip install "flama[pydantic]"
pip install "flama[database]"
pip install "flama[llm]"
pip install "flama[full]"

Usage

• •flama get --family llm --source huggingface mlx-community/gemma-4-E2B-it-qat-4bit # download & package a HuggingFace model
• •echo "What is Flama?" | flama model mlx-community_gemma-4-E2B-it-qat-4bit.flm stream --system "Be concise."
• •flama serve --model file=model.flm,url=/,name=gemma # serves HTTP API + streaming chat UI at /chat/
• •Serve multiple wire protocols simultaneously: native (/query/, /stream/, /chat/), OpenAI (/openai/v1/chat/completions), Anthropic (/anthropic/v1/messages), Ollama (/ollama/api/chat)
• •@app.mcp.tool decorator exposes a Python function as an MCP tool; JSON Schema derived from type hints

Key features

• ✓Any framework, one format: package scikit-learn, TensorFlow, PyTorch, or an LLM into a portable .flm artifact
• ✓Models on demand: download and package any model from HuggingFace Hub with one command
• ✓Every served model ships a streaming chat UI at /chat/ with Markdown, LaTeX, and Mermaid
• ✓Native first-class MCP: tools, resources, and prompts exposed via a single decorator, schemas derived from type hints
• ✓Same .flm file runs on vLLM (Linux with CUDA) or MLX (Apple Silicon), backend selected at load time
• ✓Full-featured API framework: CRUD resources over SQLAlchemy, dependency injection, auto OpenAPI/Swagger/ReDoc, SSE and NDJSON streaming, JWT auth

Best for

Turning any ML/LLM model into a production HTTP API with a built-in chat UI and MCP tools in a few commands.