PolyStrata

PolyStrata is an open-source project from VecSzn, built on Niko1221's Strata, that runs large language models on an ordinary Windows or Linux PC with a consumer graphics card. It serves nine models through two inference engines and exposes a chat app and a local API on localhost. According to the README, nothing leaves the user's PC. The latest release listed is v0.1.45-poly.1.

What It Is

PolyStrata is a local LLM inference engine and installer for mixture-of-experts models that normally need a server. The installer detects the graphics card, RAM and disk, then asks which model, size and context length to use, and whether the model should read pictures. It downloads the model files and starts a server whose app opens in the browser at http://127.0.0.1:8080. The app has a Chat tab, a live Monitor of the model and GPU/CPU/RAM, and an About page with settings.

Supported Models and Engines

Qwen3.8-Flash-Next runs on Strata's own engine, on NVIDIA or AMD cards with 12 GB of VRAM or more. Eight further models run on an engine written for PolyStrata, which the README labels experimental: GLM 5.3 Flash, DeepSeek V4 Flash, LFM2.5-24B-A2B, Qwen3.6-35B-A3B, Ornith 1.5, Gemma 4 26B-A4B, Tiel Coder 35B-A3B and MiniCPM-V 4.7 35B-A3B. Several of these read pictures, while GLM 5.3 Flash, DeepSeek V4 Flash and LFM2.5 are text-only. The server can switch between installed models while running.

How It Fits Models on Small Cards

The README explains that the busiest experts stay in GPU memory, all experts live in RAM, the processor works on the rest, and the model file stays on the SSD. Several models also use speculative decoding, in which a draft block or draft model guesses upcoming tokens and the main model verifies them. Prompts are read in large chunks. The README publishes tokens-per-second figures measured on specific laptops and workstations, and a benchmark command lets users measure their own machine.

Connecting Apps and Agents

Apps connect through an OpenAI-compatible endpoint at /v1, an Anthropic-style /v1/messages endpoint (usable with Claude Code), and a /v1/responses endpoint for Codex CLI. Reasoning effort can be set to off, low, medium or high. An MCP server lets AI tools install, start and stop PolyStrata, and an AI-setup guide lets coding assistants perform the installation. Serving to other devices requires an explicit host setting and an API key.