# ODS (Osmantic Deployment System)

> Turn your PC, Mac, or Linux box into a private AI server with one command — bundling local LLM inference, chat UI, voice, agents, workflows, RAG, and image generation.

ODS (Osmantic Deployment System) is an open-source, one-command installer that transforms any personal computer into a fully wired private AI server. Built by Osmantic and licensed under Apache 2.0, it pre-assembles and connects Ollama-compatible inference, Open WebUI, n8n, ComfyUI, voice tools, and privacy utilities so users never have to hand-wire a dozen services. The current stable release is v2.6.0, published July 28, 2026.

## What It Is

ODS is a self-hosted AI appliance framework — a category that sits between "install Ollama manually" and "pay for a cloud AI subscription." It installs a complete local AI stack in a single terminal command, auto-detects your GPU, selects the best GGUF model for your hardware memory envelope, and starts all services pre-wired to each other. No cloud account is required; prompts and data stay on the local machine unless the user explicitly enables cloud or hybrid API modes.

The stack covers every major AI workload category:

- **Chat & inference** — Open WebUI, llama-server (llama.cpp), LiteLLM gateway, TEI embeddings
- **Voice** — Whisper (speech-to-text), Kokoro (text-to-speech)
- **Agents & automation** — Hermes Agent (default local-first autonomous agent), n8n workflow automation (400+ integrations), Agent Policy Engine (APE), OpenCode browser-based coding assistant
- **Knowledge & search** — Qdrant vector database, SearXNG self-hosted search, Perplexica deep research engine
- **Image generation** — ComfyUI node-based image generation
- **Privacy & ops** — Privacy Shield PII scrubbing proxy, real-time GPU/service dashboard, Token Spy usage monitor, optional Langfuse observability

## Hardware Auto-Detection and Model Selection

The installer detects the GPU and assigns a hardware tier, then runs a versioned catalog selector to pick the best GGUF model for the detected memory envelope. Supported GPU families include NVIDIA (RTX/A-series), AMD Strix Halo unified memory, Apple Silicon (M1+ with Metal acceleration), Intel Arc (SYCL on Linux), and CPU-only fallback. Example tier picks range from Qwen3.5 2B on 8 GB CPU-only systems up to Qwen3 Coder Next on 90+ GB multi-GPU NVIDIA hosts. The Windows path uses a PowerShell catalog selector reading the same `model-library.json` as the Linux/macOS Python selector.

Bootstrap mode ships by default: a tiny 1.5B model downloads in under a minute so the user can start chatting immediately while the full model downloads in the background, then hot-swaps with zero downtime.

## Extensibility and CLI

Every service in ODS is an extension — a folder with a `manifest.yaml` and a `compose.yaml` that the dashboard, CLI, health checks, and compose stack discover automatically. Adding a custom service requires dropping a folder and running `ods enable my-service`.

The `ods` CLI manages the entire stack:

- `ods status` — health checks and GPU status
- `ods model swap T3` — switch hardware tier
- `ods mode cloud / local / hybrid` — toggle inference source
- `ods preset save / load` — snapshot and restore configurations
- `ods enable / disable` — toggle extensions

The installer itself is modular: 19 library modules, a shared service registry, and 13 ordered phases covering zero-prereq bootstrap, clean installs, product behavior, full-model capabilities, lifecycle recovery, and a User Green gate.

## Update: ODS v2.6.0

The current stable release is **v2.6.0** (published July 28, 2026). The `main` branch moves quickly and is used for active development and validation candidates. Stable patch fixes land on `release/2.6.x` before being merged forward. The README documents a release-grade validation process covering a fleet and distro lab across Ubuntu 24.04/22.04, Debian 12, Linux Mint 21.3, Fedora 41+, Rocky Linux 9, Arch Linux, Manjaro, CachyOS, and openSUSE Tumbleweed. The project lists AMD Featured Developer recognition and selection as a May 2026 AMD Lemonade Developer Challenge winner among its community acknowledgements.

## Why It Matters for Local AI

ODS addresses the gap between single-service local AI tools (Ollama, LocalAI) and a full private AI server. Where Ollama plus Open WebUI gives LLM inference and chat, ODS adds autonomous agents, workflow automation, voice, image generation, RAG, privacy tooling, and a control dashboard — all pre-wired and managed through a single CLI. The project frames self-hosted local AI as a sovereignty concern: keeping queries, data, and costs off centralized provider infrastructure.

## Features
- One-command installer for Linux, macOS, and Windows
- GPU auto-detection and model selection (NVIDIA, AMD Strix Halo, Apple Silicon, Intel Arc)
- Bootstrap mode: chat in under 2 minutes while full model downloads in background
- Open WebUI chat interface with conversation history and document upload
- llama-server (llama.cpp) high-performance local LLM inference
- LiteLLM API gateway for local/cloud/hybrid modes
- Hermes Agent: local-first autonomous agent with memory and skills
- n8n workflow automation with 400+ integrations
- Agent Policy Engine (APE) for auditing autonomous tool calls
- Whisper speech-to-text and Kokoro text-to-speech
- ComfyUI node-based local image generation
- Qdrant vector database for RAG pipelines
- SearXNG self-hosted private web search
- Perplexica deep research engine
- Privacy Shield PII scrubbing proxy
- Real-time GPU metrics and service health dashboard
- Token Spy usage monitor for local and proxied LLM traffic
- Optional Langfuse LLM observability and tracing
- Manifest-based hot-pluggable extension system
- ods CLI for full stack management
- Cloud and hybrid API modes (OpenAI, Anthropic, Together)
- Model hot-swap with automatic rollback on failure
- Multi-GPU support (NVIDIA)
- Headless setup with QR onboarding and mDNS
- Apache 2.0 open-source license

## Integrations
Ollama, Open WebUI, llama.cpp, n8n, ComfyUI, Qdrant, SearXNG, Perplexica, Whisper, Kokoro, LiteLLM, Langfuse, OpenAI API, Anthropic API, Together API, Brave Search API, Docker, Docker Compose, Slack, Hugging Face (GGUF models)

## Platforms
WINDOWS, MACOS, LINUX, WEB, API, CLI

## Pricing
Open Source

## Version
v2.6.0

## Links
- Website: https://github.com/Osmantic/ODS
- Documentation: https://github.com/Osmantic/ODS/blob/main/ods/docs/README.md
- Repository: https://github.com/Osmantic/ODS
- EveryDev.ai: https://www.everydev.ai/tools/ods-osmantic-deployment-system