Project Maya

Project Maya is an open-source local inference project created and led by Peasant Smith (@mw00) and co-maintained with @needmorevram. It runs very large mixture-of-experts models, GLM-5.3-Flash (321B parameters) and DeepSeek-V4.1-Flash-1017 (748B parameters), on hardware users already own, from one NVIDIA GPU up to 16, with experimental AMD and Windows support. The README states that nothing leaves the user's machine.

What It Is

Maya is a self-hosted inference stack made up of a model engine, a server, a browser dashboard, and quantization tools. Each supported model has its own engine. The engine keeps the most-used experts on the GPU, the next ones in RAM, and the rest on an NVMe SSD, moving them as a conversation proceeds. It measures the GPUs, CPU, RAM and SSD and tunes itself to them.

How You Use It

A setup script asks which model and quant to download, recommends one for the machine, builds the engine, and starts the server at a local address. The browser interface offers chat, a live hardware and model monitor, image attachments, and thinking levels (Off, Low, High, Max). Apps and coding agents can connect through an OpenAI-compatible API, an Anthropic-compatible messages endpoint (usable with Claude Code), and a Responses API with streaming, function tool calls and schema-validated output. Up to 8 requests can be served at once for GLM-5.3-Flash.

Models and Quants

The project publishes its own quants of GLM-5.3-Flash (Maya-S, S24, M, M-Derisked, L) with measured accuracy against the FP8 model, plus its own Maya-S quant of DeepSeek-V4.1-Flash-1017. Community GGUF and EXL3 quants can also be used. Optional features include DFlash2 drafting, KV streaming for long contexts, and a calibration step that tunes speed to the machine. The README lists user-reported benchmark numbers across many GPU configurations.

Requirements and Setup Path

The documented requirements are an NVIDIA GPU with compute capability 7.0 or newer (or an experimental AMD GPU with ROCm 7), 32 GB RAM, a fast NVMe SSD with room for the model, and Linux with CUDA 12.x, g++ and Python 3.10+. Windows 10/11 is experimental and WSL2 is not supported.