# NightRun

> A UEFI-resident LLM runtime written in Rust that boots from USB or SD card and runs a local language model entirely in RAM with no operating system underneath.

NightRun is an open-source, UEFI-resident LLM runtime written in Rust that boots directly from a USB stick or SD card, loads a quantized language model entirely into RAM, and presents a framebuffer chat interface — all without a conventional operating system. The project is available on GitHub under the MIT license and targets x86_64 PCs and the Raspberry Pi 5.

## What It Is

NightRun sits in a category of its own: it is not a Linux application, a container, or a server process. It is a single UEFI application (`BOOTX64.EFI` or `BOOTAA64.EFI`) that your machine's firmware starts directly. From that point, NightRun handles everything itself — framebuffer rendering, keyboard input, multi-core inference, and fan control on the Pi 5 — because there is no OS to delegate to. The project describes itself as "weird software" and is explicit that it stays on UEFI Boot Services rather than calling `ExitBootServices()`, which is what gives it broad hardware compatibility without shipping a kernel's worth of drivers.

## Architecture and Design Choices

The runtime is built from `no_std` Rust crates covering tensors, tokenization, model format, graphics, and UI. Key architectural decisions include:

- **Model fully in RAM**: the entire model (1.3–2.4 GB depending on selection) streams from disk into memory with inline CRC-32 verification during the read. After loading, storage is sealed — any subsequent disk read is treated as a hard fault by design.
- **Custom `.nrm` container**: NightRun converts GGUF files to its own format with a fixed header, 64-byte-aligned tensors used as zero-copy views, and an embedded tokenizer and chat template. The converter re-parses and re-checksums its own output before declaring success.
- **Hand-written SIMD kernels**: AVX2+FMA+F16C on x86_64 and NEON on the Pi 5. Q8_0, Q4_K, and Q6_K weights are used in place with no dequantized copies. The generation loop allocates nothing.
- **Offline by construction**: there is no network stack in the runtime. Nothing enters or leaves the machine after the model loads.
- **Reference-validated correctness**: greedy output is pinned token-for-token against llama.cpp for every supported model family on every change. Tokenizers are tested against fixtures generated from the official Hugging Face tokenizers.

## Supported Models and Performance

Four model configurations are validated and supported:

| Model | Quantization | Size |
|---|---|---|
| Llama 3.2 1B Instruct | Q8_0 | 1.3 GB |
| Llama 3.2 3B Instruct | Q4_K_M | 1.9 GB |
| Granite 4.1 3B | Q4_K_M | 2.0 GB |
| Qwen3 4B Instruct 2507 | Q4_K_M | 2.4 GB |

Benchmark numbers from the repository's own logs show Llama 3.2 1B reaching 52–56 tok/s prompt throughput and ~20 tok/s decode on QEMU/KVM with 8 cores and AVX2. On a real Raspberry Pi 5 (8 GB, D0 stepping), Granite 4.1 3B achieves 6.2 tok/s prompt and 3.0 tok/s decode. The project notes that decode speed slows as context fills because attention reads the full KV cache per token, and that the Pi figures predate newer ARM dot-product kernels now in the tree.

## Setup Path

The installer (`install.sh`) is the guided path. A one-liner clones the repository and launches it:

```
curl -fsSL https://nightrun.io/install-nightrun.sh | sh
```

The installer detects removable media conservatively (whole disks on USB/SD transports only, excluding anything backing `/`, `/boot`, `/home`, or swap), never preselects a device, fingerprints the chosen device before writing, and requires typing `FLASH /dev/sdX` verbatim as the final confirmation. Downloads are pinned to a Hugging Face revision and verified by SHA-256. After flashing, it reads the media back and compares digests. Prerequisites are Linux, Rust (stable + nightly), QEMU for testing without hardware, and approximately 6 GB of free disk.

## Project Status and Lineage

The GitHub repository was created in July 2026 and had 145 stars and 15 forks as of early August 2026. The project describes itself as experimental systems software: the inference engine, `.nrm` toolchain, batched prefill, installer safety logic, QEMU boots for both architectures, and Pi 5 bring-up on a real D0 board are described as solid. Still open are broad real-hardware coverage on x86 machines (firmware quirks vary), Pi 5 sustained-thermal measurements, faster NEON dot kernels awaiting board re-benchmarks, and C1-stepping Pi board testing. The project also notes it was built primarily with Claude Code using the Fable 5 model, framing NightRun as a test of how far a coding agent can be pushed on a bootable systems project with hard correctness requirements.

## Features
- UEFI-resident runtime, no OS required
- Boots from USB (x86_64) or SD card (Raspberry Pi 5)
- Full model RAM residency with inline CRC-32 verification
- Storage sealed after model load — no disk I/O during inference
- Hand-written AVX2+FMA+F16C (x86_64) and NEON (Pi 5) quantized kernels
- Supports Q8_0, Q4_K, and Q6_K quantizations
- Custom .nrm model container format with zero-copy tensor views
- Batched prompt prefill (up to 64 tokens per pass)
- Framebuffer chat UI with scrollback, caret editing, and live stats
- Token-for-token parity validation against llama.cpp
- Tokenizer parity tested against Hugging Face official tokenizers
- Multi-core inference via UEFI firmware MP services
- Raspberry Pi 5 fan control via RP1
- Offline by construction — no network stack
- Interactive installer with paranoid disk-safety logic
- QEMU/KVM testing support before flashing hardware
- Supports Llama 3.2, Granite 4.1, and Qwen3 model families
- no_std Rust codebase

## Integrations
UEFI firmware (OVMF/EDK2), QEMU/KVM, Hugging Face model hub (download/verification), llama.cpp (reference validation), Raspberry Pi 5 UEFI firmware, GGUF model format (via nrconvert), Spleen bitmap font

## Platforms
WINDOWS, LINUX, WEB, API, CLI

## Pricing
Open Source

## Links
- Website: https://nightrun.io
- Documentation: https://nightrun.io/docs.html
- Repository: https://github.com/hardrave/NIGHTRUN
- EveryDev.ai: https://www.everydev.ai/tools/nightrun