Subtext

Subtext is an open-source tool built by ninjahawk that applies the Jacobian lens method continuously during live conversation with a local language model, making the model's intermediate reasoning steps directly watchable in real time. It runs locally on consumer hardware (NVIDIA GPU or Apple Silicon) and renders internal activations as they happen — both while the model reads the user's message and while it generates its reply. The project is licensed under Apache 2.0 and is independent of Anthropic, though it builds on Anthropic's reference implementation.

What It Is

Subtext is a mechanistic interpretability instrument — a tool for observing what a language model is "thinking" internally before and during its output. It uses the Jacobian lens, a technique identified in recent Anthropic research, which transports residual-stream activations at any layer into the final-layer basis and decodes them through the model's own unembedding. The result answers: which vocabulary words is this internal state disposed to produce, now or later? Subtext applies this lens at nine depths on every token, streaming the results live in a browser-based canvas.

How the Lens Works in Practice

The tool reveals a consistent gap between a model's internal state and its visible output. Three phenomena documented in the demo session illustrate this:

• Verdict before reply: While the model is still reading the user's message, the workspace already holds relevant concepts — before any output token is emitted.
• Judgment formed, then verbalized: The model's internal conclusion (e.g., incorrect) dominates the workspace several tokens before the corresponding words appear in output.
• Plans held while other words are spoken: Mid-explanation, the workspace holds upcoming technical terms while the current output token is unrelated — demonstrating lookahead planning.

These reproduce, on an open 4B model on consumer hardware, phenomena described in Anthropic's paper that originally used Claude-scale models.

Architecture and Setup

Subtext runs as a local Python server (server.py) communicating with a single-file browser frontend over WebSocket. The default model is Qwen3.5-4B (bf16, HuggingFace Transformers, KV cache), paired with a pre-fitted Jacobian lens published by Neuronpedia. Per token, residual hooks at 9 layers feed into the lens transport, unembedding, and a full-vocabulary softmax, with word-start top-k results streamed to the browser.

System requirements:

• NVIDIA GPU with ~10 GB VRAM (CUDA), or Apple Silicon Mac with 16 GB+ unified memory (macOS 14+, PyTorch ≥ 2.3)
• Python 3.11+
• ~9 GB download for model and lens weights on first launch

Setup is straightforward:

• Clone the repo, install requirements, run server.py
• Windows users can use start.bat or the UTF-8 flag
• Sessions can be exported as JSON and replayed in any browser without a GPU

Display and Interaction

The browser canvas renders each lens readout as a word, where vertical position corresponds to layer depth (early/perception layers at top, emission-approaching layers at bottom), and size/opacity encode readout strength. Amber marks readouts taken during the reading phase; blue marks the generation phase. Users can hover over words to see per-layer activation profiles, or click to open an inspector showing peak strength, mean depth, and strength history. A right-side panel records the conversation, a live ranking of active readouts, and a per-token ledger.

Limitations and Scope

The instrument inherits the method's known limitations: the lens only reads concepts corresponding to single vocabulary tokens, so multi-token concepts are invisible or fragmentary. It captures the workspace identified in the Anthropic paper, not the entirety of the model's internal state, and layers below the fitted range are unobserved. The README explicitly notes that workspace readouts demonstrate functional availability of information for report and reasoning — they do not demonstrate subjective experience.