LLMxRay

LLMxRay is an open-source local LLM observatory by LogneBudo (Ivan Stankovic) that sits on top of a locally running Ollama instance. It lets users chat with models while seeing token streaming, quality flags, timing and throughput, and adds benchmarking, comparison, embeddings and RAG inspection tools. The latest release listed is v0.7.2, which adds a Live page.

What It Is

LLMxRay is a browser-based interface for inspecting how local models behave. It connects to Ollama (the project states it is tested against 0.40.x) and stores data in the browser's IndexedDB. The project states there are no cloud calls, API keys or accounts.

Core tools include Chat Diagnostics with per-token streaming and thinking-block visualization, automatic quality badges (repetition, refusal, gibberish, empty, truncation), a comparison workbench with up to four slots, and a Surgical Benchmark that scores models using real token logprobs through the OpenAI-compatible endpoint. Results can be exported as JSON, CSV, Markdown, JSONL and TXT.

Live, Cache Lab and Decision Lab

The Live page, new in 0.7.2, opens a second port (11435) that behaves like Ollama and forwards requests to it. Pointing tools such as VS Code Copilot, Open WebUI or Continue at it shows each turn's prompt size, cached tokens, prefill and generation speed, tool calls, and where the prompt stopped matching the previous turn. Only counts and timings are kept in memory; prompt text is opt-in.

Cache Lab finds values that change between turns and breaks KV cache reuse, then measures the effect of moving them. Decision Lab queries Ollama decision models and shows the probability distribution behind each answer.

Building and Data Tools

The Embeddings Lab visualizes vectors and measures cosine similarity, and a local knowledge base chunks PDFs, DOCX and CSV files so users can see what RAG retrieves. The Tool Workshop provides a visual node canvas with bidirectional code sync for building tool definitions, probing APIs and testing. Protocol Observatory compares replies across Ollama's native, OpenAI-compatible and Anthropic-compatible APIs, and a Fill-in-the-Middle playground supports code completion models. Accepted or rejected outputs can be curated into JSONL training data.

Setup Path

Run it with npx llmxray or the Docker image, then open localhost:5174. Ollama must be running with at least one model pulled. It can also be run from source with Node. The interface is available in English, French, Chinese, Arabic and Serbian.