Announcing Miles v0.1

This week, we launched Miles v0.1, an open-source RL framework for LLMs and multimodal models.

RL training is easy to start but hard to debug. Rollouts can take hours. Tool calls add unpredictable latency. Small inconsistencies between inference and training can quietly invalidate an experiment.

Our answer is Miles.

Miles is an open RL framework to make training simple, efficient, and stable.

Performance

Miles combines fast multi-turn rollouts with SGLang, fully asynchronous RL, and Token-In-Token-Out (TITO). For MoE models, Rollout Routing Replay (R3) also keeps expert routing consistent between rollout and training.

Miles pairs SGLang with Megatron-LM or FSDP2 for training. It supports low-precision training and rollout, CPU and NVMe optimizer offload, and fast P2P weight synchronization between training and inference workers.

Flexibility

Miles runs on both NVIDIA and AMD GPUs and supports models ranging from small dense models to frontier-scale MoEs. In one reference run, Miles trained GLM-5.2 744B on terminal-use tasks with fully async RL across 64 NVIDIA GB300 GPUs, with rollout and training almost fully overlapped.

Miles is also customizable. Rollouts, environments, rewards, losses, data selection, and training backends live behind clean interfaces. It ships with recipes for RL, SFT, LoRA, and on-policy distillation, along with integrations for agent environments and sandbox providers. New algorithms can be implemented with minimal effort.

Real workloads shape Miles

“Miles is the best RL framework for bridging the gap between the digital and physical worlds, offering strong efficiency, scalability, and flexibility.”Byron HsuPeriodic Labs“Miles has been rock-solid for us as we've tackled RL problems across coding, computer use, and non-verifiable domains. Miles is powerful while still remaining stable during long runs.”Peyton WaltersModal“Miles combines a high-performance foundation for RL training at scale with an ambitious technical vision. Together, we're turning advances in algorithms and engineering into practical capabilities for demanding RL workloads.”Ziang Lihumans&“Miles removes much of the engineering complexity from agentic RL, letting us iterate faster and focus on improving the quality of our agents and RL environment data.”Guohao LiEigent.AI and CAMEL-AI“Miles gives us the best of both worlds for RL: a simple abstraction for the common case and direct access to low-level primitives when we need more control.”Cyrus AsgariDecagon

“Miles has been a powerful framework for our agentic RL work. Its efficiency, robustness, and advanced features for large-scale models allowed us to iterate faster and scale experiments with ease.”Rameswar PandaIBM

“Miles is an extremely feature-rich framework that allows us to run various post-training workloads up to trillion-parameter model scale. It does so without needing to sacrifice flexibility, so we can continue to iterate and train quickly.”Kasey ZhangOsmosis

Periodic Labs uses Miles to train trillion-parameter models across thousands of GPUs with extremely long-horizon rollouts for scientific RL. Together, we improved rollout throughput by 3×, weight synchronization by 10×, and weight conversion by 30×, with everything upstreamed.

Miles v0.1 is open source and available now. Over the past nine months, 72 contributors have landed more than 1,300 commits, backed by extensive GPU end-to-end testing across frontier models and hardware.

Learn more about Miles, explore the documentation, read the technical blog, and contribute to GitHub.

We’re excited to see what you build with Miles.

The RadixArk Team