NightRun

NightRun is an open-source, UEFI-resident LLM runtime written in Rust that boots directly from a USB stick or SD card, loads a quantized language model entirely into RAM, and presents a framebuffer chat interface — all without a conventional operating system. The project is available on GitHub under the MIT license and targets x86_64 PCs and the Raspberry Pi 5.

What It Is

NightRun sits in a category of its own: it is not a Linux application, a container, or a server process. It is a single UEFI application (BOOTX64.EFI or BOOTAA64.EFI) that your machine's firmware starts directly. From that point, NightRun handles everything itself — framebuffer rendering, keyboard input, multi-core inference, and fan control on the Pi 5 — because there is no OS to delegate to. The project describes itself as "weird software" and is explicit that it stays on UEFI Boot Services rather than calling ExitBootServices(), which is what gives it broad hardware compatibility without shipping a kernel's worth of drivers.

Architecture and Design Choices

The runtime is built from no_std Rust crates covering tensors, tokenization, model format, graphics, and UI. Key architectural decisions include:

• Model fully in RAM: the entire model (1.3–2.4 GB depending on selection) streams from disk into memory with inline CRC-32 verification during the read. After loading, storage is sealed — any subsequent disk read is treated as a hard fault by design.
• Custom .nrm container: NightRun converts GGUF files to its own format with a fixed header, 64-byte-aligned tensors used as zero-copy views, and an embedded tokenizer and chat template. The converter re-parses and re-checksums its own output before declaring success.
• Hand-written SIMD kernels: AVX2+FMA+F16C on x86_64 and NEON on the Pi 5. Q8_0, Q4_K, and Q6_K weights are used in place with no dequantized copies. The generation loop allocates nothing.
• Offline by construction: there is no network stack in the runtime. Nothing enters or leaves the machine after the model loads.
• Reference-validated correctness: greedy output is pinned token-for-token against llama.cpp for every supported model family on every change. Tokenizers are tested against fixtures generated from the official Hugging Face tokenizers.

Supported Models and Performance

Four model configurations are validated and supported:

Benchmark numbers from the repository's own logs show Llama 3.2 1B reaching 52–56 tok/s prompt throughput and ~20 tok/s decode on QEMU/KVM with 8 cores and AVX2. On a real Raspberry Pi 5 (8 GB, D0 stepping), Granite 4.1 3B achieves 6.2 tok/s prompt and 3.0 tok/s decode. The project notes that decode speed slows as context fills because attention reads the full KV cache per token, and that the Pi figures predate newer ARM dot-product kernels now in the tree.

Setup Path

The installer (install.sh) is the guided path. A one-liner clones the repository and launches it:

curl -fsSL https://nightrun.io/install-nightrun.sh | sh

The installer detects removable media conservatively (whole disks on USB/SD transports only, excluding anything backing /, /boot, /home, or swap), never preselects a device, fingerprints the chosen device before writing, and requires typing FLASH /dev/sdX verbatim as the final confirmation. Downloads are pinned to a Hugging Face revision and verified by SHA-256. After flashing, it reads the media back and compares digests. Prerequisites are Linux, Rust (stable + nightly), QEMU for testing without hardware, and approximately 6 GB of free disk.

Project Status and Lineage

The GitHub repository was created in July 2026 and had 145 stars and 15 forks as of early August 2026. The project describes itself as experimental systems software: the inference engine, .nrm toolchain, batched prefill, installer safety logic, QEMU boots for both architectures, and Pi 5 bring-up on a real D0 board are described as solid. Still open are broad real-hardware coverage on x86 machines (firmware quirks vary), Pi 5 sustained-thermal measurements, faster NEON dot kernels awaiting board re-benchmarks, and C1-stepping Pi board testing. The project also notes it was built primarily with Claude Code using the Fable 5 model, framing NightRun as a test of how far a coding agent can be pushed on a bootable systems project with hard correctness requirements.