TensorFold

TensorFold, made by TensorFold Labs Ltd, is an Apache-2.0 inference engine that serves language models from a Zig binary on Apple Silicon and NVIDIA GPUs. The engine reads checkpoints, tokenizes requests and runs Metal or CUDA kernels directly, so serving needs no Python or MLX installation. It ships as a CLI, a Mac desktop app and an enterprise platform called Starlight.

What It Is

TensorFold is a local model server. You pull a checkpoint from Hugging Face with tensorfold pull, start it with tensorfold serve, and send requests to an HTTP endpoint that supports chat completions, completions, Responses and Anthropic Messages. /v1/models lists the served name. Installation is through Homebrew on macOS or a release archive on Linux with an NVIDIA GPU, and a native Windows build guide is also referenced.

Models and Serving Options

The README documents serving paths for model families including Qwen, Nemotron, GLM, LFM, Gemma and others, with per-model notes on hardware scope, such as one Mac, one GB10, or two M5 Ultras for some GLM setups. Serve flags cover context length, parallel streams, API keys, sampling defaults, reasoning controls, external drafters, prompt caching and speech options. Model weights carry their own licenses.

Decoding, Caching and Memory Features

On supported models a drafter proposes several tokens and the target model checks them together, committing only the matching prefix. The README states that every accepted draft must equal the token the engine would produce without drafting. Retained prompt prefixes can be reused, and optional context compaction summarizes older turns into a memory note instead of refusing an overflowing conversation. Optional Cortheon mode adds a reasoning program and server-run web research. Living Weights is an experimental feature that writes learned facts into a model's checkpoint files.

Desktop, Enterprise and Benchmarks

The Mac desktop app bundles the engine, model downloads, hardware recommendations, live generation metrics, coding workers and document editing. TensorFold Starlight is the enterprise layer: one governed endpoint for a fleet of engines, with OIDC or SAML sign-in, SCIM provisioning, model placement and token accounting. The site lists recorded benchmark runs on named hardware, noting that each figure is a separate release measurement rather than a like-for-like comparison, and offers a community benchmark leaderboard.