# MAFDET

> Local LLM chat app that reads and reasons about your message while you type, so the answer starts sooner after Enter.

MAFDET is an open-source, Apache-2.0 licensed local LLM chat app published under the GitHub handle TheLastOxygen. It prefills the model's KV cache and starts reasoning on your message while you are still typing, so less work remains after you press Enter. The project is labeled a prototype and runs on llama.cpp, with a reference machine of an RTX 2060 (6 GB VRAM) and a CPU-only mode.

## What It Is

MAFDET is a desktop and terminal chat front end for locally run language models. Instead of waiting for Enter, it tokenizes the current text on every pause, holds back the last few tokens, finds the longest common prefix with the cached tokens, trims the cache there and prefills only the new part. Edits, backspaces and pasted blocks are handled by that prefix check, which brings the cache back to what a from-scratch computation would produce.

## Thinking ahead

For Qwen3 in thinking mode, the model also reasons in short 32-token chunks while you type. When the text changes, the earlier reasoning stays in the prompt as notes after the new message version, and on Enter the model finishes reasoning and answers. This part is not exact, since the model reasons on intermediate versions of the message. The README reports that on Qwen3-4B with 24 word problems typed in real time, mean wait after Enter fell from 12.2 s to 3.2 s with 24/24 correct answers both ways, and on Qwen3-14B from 108 s to 47 s on 6 problems. If you pause and send the message unchanged, it can also write the exact answer ahead.

## App, models and measurement

The app comes as a window (`mafdet gui`) and a terminal app (`mafdet tui`), with a model catalog (`mafdet models`) and a hardware check (`mafdet doctor`). The catalog covers Qwen3, Llama 3, Gemma 3, Mistral and Phi-4 from 0.6B to 32B parameters, with per-machine speed estimates and a Measure benchmark. Commands `hunt`, `race` and `judge` simulate typing, benchmark prefill and decode, and verify results against a from-scratch computation. The interface is available in English, Italian, French and German.

## Setup path

On Windows, double-click `start_mafdet.bat`, or install with pip from a clone and run `mafdet`. First start detects the hardware, installs a suitable llama.cpp build (CUDA or CPU) and downloads a fitting model. Linux code paths exist but are untested; macOS and AMD GPU support are on the roadmap.

## Features
- Incremental prefill with common-prefix KV cache sync while typing
- Token hold-back for partially typed words
- Reasoning ahead for Qwen3 kept across edits
- Exact answer written ahead when message is unchanged
- Desktop window and terminal app interfaces
- Model catalog with per-machine speed estimates and Measure benchmark
- Automatic llama.cpp install (CUDA or CPU) and model download
- Typing simulation, benchmark and correctness-check commands
- Optional speculative decoding with recycled drafts or a draft model
- UI in English, Italian, French and German

## Integrations
llama.cpp, Hugging Face, Qwen3, Llama 3, Gemma 3, Mistral, Phi-4, PyTorch

## Platforms
WINDOWS, MACOS, LINUX, API, CLI

## Pricing
Open Source

## Version
v0.1.0

## Links
- Website: https://github.com/TheLastOxygen/mafdet
- Repository: https://github.com/TheLastOxygen/mafdet
- EveryDev.ai: https://www.everydev.ai/tools/mafdet