OLMo

OLMo (Open Language Model) is a fully open-source large language model project from the Allen Institute for AI (Ai2), released under the Apache 2.0 license. It provides not just model weights but also training code, data, and evaluation tools — making it one of the most transparent LLM releases available. The project is designed by scientists, for scientists, and targets researchers and developers who need full access to the entire model development pipeline.

What It Is

OLMo is a repository and model family for training and using Ai2's open language models. Unlike many "open" models that release only weights, OLMo releases the full training codebase, pretraining data, intermediate checkpoints, and evaluation frameworks. The current generation is OLMo-2, which comes in 1B, 7B, 13B, and 32B parameter variants, all available on Hugging Face in both base and instruction-tuned forms.

Two-Stage Pretraining Architecture

OLMo-2 uses a two-stage training procedure:

• Stage 1: Large-scale pretraining on 4–5 trillion tokens of mostly web-based data (OLMo-mix-1124 dataset)
• Stage 2: Continued training on 50–300 billion tokens of high-quality, targeted data (Dolmino-mix-1124 dataset)

For the 7B and 13B models, Ai2 trains multiple runs with different random seeds and then averages ("soups") the resulting model weights to produce the final checkpoint. All intermediate checkpoints — at minimum every 1,000 training steps — are publicly released.

Update: OLMo-2 and the Move to OLMo-core

The original OLMo repository (github.com/allenai/OLMo) is now marked as out of date and no longer actively maintained. The latest releases, including the OLMo-2 32B model, are developed and maintained in the newer OLMo-core repository (github.com/allenai/OLMo-core). The most recent model variant is OLMo-2-0425-1B, released in April 2025. The legacy repository's latest release tag is v0.6.0 (December 2024).

Inference and Deployment

OLMo models integrate directly with the Hugging Face Transformers library, enabling standard AutoModelForCausalLM and pipeline-based inference. The repository also includes an example script for hosting OLMo-2 on Modal.com with an OpenAI-compatible API endpoint. Quantization via bitsandbytes (8-bit) is supported. Models can be run on CUDA GPUs or Apple Silicon (Mac) devices with PyTorch 2.5+.

Evaluation and Ecosystem

Ai2 maintains separate repositories for evaluation: OLMo-eval and olmes. Instruction-tuned variants (Instruct models) are available for all four size classes. The project also connects to Ai2's broader open model ecosystem, including Tülu 3 (post-training), Molmo (multimodal), and the Ai2 Playground for interactive access.

Why It Matters for Researchers

OLMo's distinguishing characteristic is its commitment to full openness — training configs, WandB training logs, data provenance CSVs, and every checkpoint are publicly accessible. This level of transparency is explicitly positioned by Ai2 as enabling reproducible science and community-driven advancement of language model research, in contrast to models that release only final weights.