MAFDET

MAFDET is an open-source, Apache-2.0 licensed local LLM chat app published under the GitHub handle TheLastOxygen. It prefills the model's KV cache and starts reasoning on your message while you are still typing, so less work remains after you press Enter. The project is labeled a prototype and runs on llama.cpp, with a reference machine of an RTX 2060 (6 GB VRAM) and a CPU-only mode.

What It Is

MAFDET is a desktop and terminal chat front end for locally run language models. Instead of waiting for Enter, it tokenizes the current text on every pause, holds back the last few tokens, finds the longest common prefix with the cached tokens, trims the cache there and prefills only the new part. Edits, backspaces and pasted blocks are handled by that prefix check, which brings the cache back to what a from-scratch computation would produce.

Thinking ahead

For Qwen3 in thinking mode, the model also reasons in short 32-token chunks while you type. When the text changes, the earlier reasoning stays in the prompt as notes after the new message version, and on Enter the model finishes reasoning and answers. This part is not exact, since the model reasons on intermediate versions of the message. The README reports that on Qwen3-4B with 24 word problems typed in real time, mean wait after Enter fell from 12.2 s to 3.2 s with 24/24 correct answers both ways, and on Qwen3-14B from 108 s to 47 s on 6 problems. If you pause and send the message unchanged, it can also write the exact answer ahead.

App, models and measurement

The app comes as a window (mafdet gui) and a terminal app (mafdet tui), with a model catalog (mafdet models) and a hardware check (mafdet doctor). The catalog covers Qwen3, Llama 3, Gemma 3, Mistral and Phi-4 from 0.6B to 32B parameters, with per-machine speed estimates and a Measure benchmark. Commands hunt, race and judge simulate typing, benchmark prefill and decode, and verify results against a from-scratch computation. The interface is available in English, Italian, French and German.

Setup path

On Windows, double-click start_mafdet.bat, or install with pip from a clone and run mafdet. First start detects the hardware, installs a suitable llama.cpp build (CUDA or CPU) and downloads a fitting model. Linux code paths exist but are untested; macOS and AMD GPU support are on the roadmap.