Miles
Open-source reinforcement learning framework for large-scale LLM and VLM post-training, pairing SGLang rollout with Megatron-LM training.
At a Glance
Self-hosted Miles RL framework released under the Apache License 2.0; hardware and infrastructure costs are external.
Engagement
Available On
Updated Oct 2026
About Miles
Miles is an open-source reinforcement learning framework for large-scale model post-training, developed under the radixark GitHub organization and released under the Apache 2.0 license. It pairs SGLang for high-throughput rollout with Megatron-LM for scalable training, and also supports FSDP2 and torchtitan backends. The latest GitHub release listed is v0.1.2.
What It Is
Miles is a post-training framework for LLMs, VLMs and diffusion models, with a focus on reinforcement learning and agentic workloads. Users launch recipe scripts that run rollout and training on infrastructure they control, on Ray or as a Helm release on Kubernetes. Recipes cover GRPO, GSPO, PPO, REINFORCE++, supervised fine-tuning and on-policy distillation, in full-parameter or LoRA form. Rewards, advantage computation, loss and data selection are customizable.
Agentic Rollout and Environments
Miles manages full trajectories of model responses, tool calls, observations and environment interactions. Its token-in-token-out (TITO) data path preserves the exact tokens produced during rollout, including with black-box agent harnesses. Connectors are provided for Harbor, HUD, NeMo Gym, OpenEnv and Verifiers, with task sandboxes on AgentENV, Daytona, E2B or Modal. Synchronous and fully asynchronous RL are both supported.
Performance and Stability Features
The docs describe decoupled rollout and training workers, P2P RDMA weight updates, and MXFP8, NVFP4, FP8 and INT4 QAT low-precision training. Rollout Routing Replay (R3) replays MoE expert routing in the trainer's forward pass. Fault tolerance recovers a failed SGLang engine and resumes the run in place. LoRA and multi-LoRA adapters load directly into SGLang for rollout.
Models and Hardware
Recipes exist for DeepSeek, Kimi, GLM, Qwen, Nemotron, Gemma, MiMo, JoyAI, gpt-oss and Inkling. Diffusion recipes (Flow-GRPO, DiffusionNFT, SFT) live in the companion Miles-diffusion project. Supported hardware is NVIDIA A100 through GB300 and AMD MI300X through MI355X via ROCm. The docs recommend a Docker-based quick start that runs a complete GRPO loop.
Community Discussions
Be the first to start a conversation about Miles
Share your experience with Miles, ask questions, or help others learn from your insights.
Pricing
Open Source (Apache-2.0)
Self-hosted Miles RL framework released under the Apache License 2.0; hardware and infrastructure costs are external.
- Apache License 2.0
- Fully async RL with SGLang rollout
- Megatron-LM, FSDP2 and torchtitan training backends
- SFT, RL and on-policy distillation recipes
- LoRA and multi-LoRA support
Capabilities
Key Features
- Fully asynchronous RL with decoupled rollout and training
- SGLang-based agentic rollout with router
- Token-in-Token-Out (TITO) data path
- Rollout Routing Replay (R3) for MoE
- P2P RDMA weight updates
- MXFP8, NVFP4, FP8 and INT4 QAT low-precision training
- LoRA and multi-LoRA
- Fault tolerance with in-place engine recovery
- GRPO, GSPO, PPO, REINFORCE++, SFT and on-policy distillation recipes
- Megatron-LM, FSDP2 and torchtitan training backends
- Ray and Kubernetes (Helm) deployment
- Day-0 recipes for frontier open models
- Diffusion model post-training via Miles-diffusion
