wllama

wllama is an open-source WebAssembly binding for llama.cpp, created and maintained by Xuan-Son Nguyen. It lets web applications run GGUF language models directly in the browser, with no backend required, and is distributed as the npm package @wllama/wllama. A hosted demo app is available on Hugging Face Spaces.

What It Is

wllama is a TypeScript library that compiles llama.cpp to WebAssembly so LLM inference can run client-side. Developers load a GGUF model from the Hugging Face hub or from a URL, then call an OpenAI-compatible API to create chat completions or embeddings. Inference runs inside a worker so it does not block UI rendering.

Capabilities

The V3 release adds WebGPU, multimodal (image and audio file input) and tool calling support. With WebGPU, all layers are offloaded to the GPU by default, and the n_gpu_layers parameter can adjust or disable this. The library automatically switches between single-thread and multi-thread builds based on browser support, and has no runtime dependencies. Examples cover completions, embeddings with cosine distance, multimodal completion, tool calling and decision models.

Model Handling and Limits

Because of ArrayBuffer size restrictions, files are limited to 2GB, so larger models must be split with llama-gguf-split. Splitting into chunks of up to 512MB also allows parallel downloads. Quantized Q4, Q5 or Q6 models are recommended. The project notes that WebAssembly overhead can reduce performance by 25% to 50% compared with native llama.cpp, that smartphones may be buggy, and that Safari is not supported because it lacks Memory64 support. Multi-threading requires Cross-Origin-Embedder-Policy and Cross-Origin-Opener-Policy headers.

Setup Path

Install with npm, import the Wllama class, point it at the wasm file paths, load a model and request a chat completion. The wasm binaries can also be built from source using Docker.