ODS (Osmantic Deployment System)

ODS (Osmantic Deployment System) is an open-source, one-command installer that transforms any personal computer into a fully wired private AI server. Built by Osmantic and licensed under Apache 2.0, it pre-assembles and connects Ollama-compatible inference, Open WebUI, n8n, ComfyUI, voice tools, and privacy utilities so users never have to hand-wire a dozen services. The current stable release is v2.6.0, published July 28, 2026.

What It Is

ODS is a self-hosted AI appliance framework — a category that sits between "install Ollama manually" and "pay for a cloud AI subscription." It installs a complete local AI stack in a single terminal command, auto-detects your GPU, selects the best GGUF model for your hardware memory envelope, and starts all services pre-wired to each other. No cloud account is required; prompts and data stay on the local machine unless the user explicitly enables cloud or hybrid API modes.

The stack covers every major AI workload category:

• Chat & inference — Open WebUI, llama-server (llama.cpp), LiteLLM gateway, TEI embeddings
• Voice — Whisper (speech-to-text), Kokoro (text-to-speech)
• Agents & automation — Hermes Agent (default local-first autonomous agent), n8n workflow automation (400+ integrations), Agent Policy Engine (APE), OpenCode browser-based coding assistant
• Knowledge & search — Qdrant vector database, SearXNG self-hosted search, Perplexica deep research engine
• Image generation — ComfyUI node-based image generation
• Privacy & ops — Privacy Shield PII scrubbing proxy, real-time GPU/service dashboard, Token Spy usage monitor, optional Langfuse observability

Hardware Auto-Detection and Model Selection

The installer detects the GPU and assigns a hardware tier, then runs a versioned catalog selector to pick the best GGUF model for the detected memory envelope. Supported GPU families include NVIDIA (RTX/A-series), AMD Strix Halo unified memory, Apple Silicon (M1+ with Metal acceleration), Intel Arc (SYCL on Linux), and CPU-only fallback. Example tier picks range from Qwen3.5 2B on 8 GB CPU-only systems up to Qwen3 Coder Next on 90+ GB multi-GPU NVIDIA hosts. The Windows path uses a PowerShell catalog selector reading the same model-library.json as the Linux/macOS Python selector.

Bootstrap mode ships by default: a tiny 1.5B model downloads in under a minute so the user can start chatting immediately while the full model downloads in the background, then hot-swaps with zero downtime.

Extensibility and CLI

Every service in ODS is an extension — a folder with a manifest.yaml and a compose.yaml that the dashboard, CLI, health checks, and compose stack discover automatically. Adding a custom service requires dropping a folder and running ods enable my-service.

The ods CLI manages the entire stack:

• ods status — health checks and GPU status
• ods model swap T3 — switch hardware tier
• ods mode cloud / local / hybrid — toggle inference source
• ods preset save / load — snapshot and restore configurations
• ods enable / disable — toggle extensions

The installer itself is modular: 19 library modules, a shared service registry, and 13 ordered phases covering zero-prereq bootstrap, clean installs, product behavior, full-model capabilities, lifecycle recovery, and a User Green gate.

Update: ODS v2.6.0

The current stable release is v2.6.0 (published July 28, 2026). The main branch moves quickly and is used for active development and validation candidates. Stable patch fixes land on release/2.6.x before being merged forward. The README documents a release-grade validation process covering a fleet and distro lab across Ubuntu 24.04/22.04, Debian 12, Linux Mint 21.3, Fedora 41+, Rocky Linux 9, Arch Linux, Manjaro, CachyOS, and openSUSE Tumbleweed. The project lists AMD Featured Developer recognition and selection as a May 2026 AMD Lemonade Developer Challenge winner among its community acknowledgements.

Why It Matters for Local AI

ODS addresses the gap between single-service local AI tools (Ollama, LocalAI) and a full private AI server. Where Ollama plus Open WebUI gives LLM inference and chat, ODS adds autonomous agents, workflow automation, voice, image generation, RAG, privacy tooling, and a control dashboard — all pre-wired and managed through a single CLI. The project frames self-hosted local AI as a sovereignty concern: keeping queries, data, and costs off centralized provider infrastructure.