TL;DR
Miles v0.1 describes a full-stack post-training system spanning rollout generation, optimization, and weight synchronization. Its reference run demonstrates asynchronous terminal-agent RL with a 744B-parameter MoE model and makes the system’s supported configurations explicit.
Why it matters
Scaling RL requires coordinating generation, environment execution, training, and synchronization. A concrete system report helps explain how numerical fidelity and resource scheduling determine whether theoretical compute capacity becomes useful training throughput.
Key findings
- 01
The system builds on slime with SGLang rollouts, Megatron-LM or FSDP training, and multiple weight-update transports. It also supports LoRA RL, on-policy distillation, supervised fine-tuning, and an extension to diffusion models.
- 02
Token-in-token-out trajectories preserve sampled token identifiers, while rollout routing replay records MoE expert choices. These mechanisms address mismatches that can arise when multi-turn trajectories are reconstructed or processed by different engines.
- 03
The reference case trains GLM-5.2, with 744B total and 40B active parameters, on terminal-use tasks using 64 GB300 GPUs split equally between rollout and training. The run uses asynchronous scheduling and supports sessions up to 65,536 tokens.
- 04
The median training step is 263 seconds over the first 30 measured steps. Across the single 100-step run, reported rollout–training divergence averages 0.0369; raw reward’s nine-step moving average rises from 0.438 to 0.556, which the authors explicitly report as an observation rather than a measured improvement.
Evidence & limits
The throughput figure belongs to one hardware, model, and task configuration. The 1,042-second warm-up at step zero is clipped from the plotted range, and the reported median does not summarize all end-to-end setup costs.
A single 100-step run cannot separate reward changes from run-to-run variation. The case study does not establish a general capability gain or a universal speed advantage over other RL systems.
Supported scope varies by component. The report identifies incomplete vision-language session support, early-stage precision formats, and weight-transfer paths covering only some model families.
Scaling dimensions
Models, methods & benchmarks
- Models
- GLM-5.2
- Algorithms
- Asynchronous RL, Rollout routing replay, Token-in-token-out, On-Policy Distillation
- Benchmarks
- Terminal-use coding tasks
Topics