The RL Scaling Bottleneck Map
RL Scaling
A practical framework for locating the binding constraint in an RL system across environments, rollout generation, reward quality, optimization, and evaluation.
Research index
What is actually
moving the frontier?
Source-linked research notes with original summaries and a clear view of why each development matters for scaling intelligent systems.
22 items
GLM-5.3: Frontier Coding with Emergent Cyber Capabilities
Z.ai
An official GLM-5.3 release note describing a post-training-only update on the same base model as GLM-5.2, with expanded executable environments for complex coding, security, and long-horizon professional workflows.
Four LLM loss functions → four flavors of LLM misalignment
Steven Byrnes
A community synthesis linking four training signals—imitation, human approval, automatic verifiers, and LLM judges—to distinct classes of observed or hypothesized alignment failure.
RLVR that rewards red teaming the training environment
Fiora Starlight
A LessWrong proposal to let policies report exploitable RLVR environment bugs after a rollout, reward high-quality verified reports, and use them to patch the training environment.
Kimi K3: Open Frontier Intelligence
Moonshot AI
A technical report on Kimi K3, a 2.8T-parameter sparse multimodal model with 104B active parameters and a one-million-token context window, post-trained with reinforcement learning across general, agentic, coding, and reasoning domains.
Athena-Brain Technical Report: An Efficient Robot Brain for General Intelligence and Embodied Interaction
Jialian Li
A technical report on Athena-Brain-8B, an on-device embodied model trained through general supervised fine-tuning, general reinforcement learning, embodied-expert training, and model merging.
GR2 Technical Report
Yufei Li
A production-oriented technical report on a generative reasoning re-ranker trained with semantic-ID mid-training, teacher-trace distillation, on-policy distillation, and reinforcement learning from verifiable ranking rewards.
GLM-5.2: Built for Long-Horizon Tasks
Z.ai
An official release of the 750B-A40B GLM-5.2 model, combining a one-million-token context, IndexShare sparse-attention reuse, and larger-scale agentic reinforcement learning for long-horizon tasks.
Ling and Ring 2.6 Technical Report: Efficient and Instant Agentic Intelligence at Trillion-Parameter Scale
Ang Li
A model-family report on Ling-2.6 and Ring-2.6, combining architectural migration, long-context efficiency work, token-efficient reasoning objectives, and asynchronous agent reinforcement learning at trillion-parameter scale.
The MiniMax-M2 Series: Mini Activations Unleashing Max Real-World Intelligence
MiniMax
A technical report on the MiniMax-M2 family, pairing a 229.9B-parameter sparse MoE with agent-generated, verifiable trajectories and Forge, a scalable reinforcement-learning system for long-horizon agents.
GLM-5.1: Towards Long-Horizon Tasks
Z.ai
An official GLM-5.1 model update focused on keeping an agent productive across longer coding and engineering runs through repeated execution, inspection, diagnosis, and strategy revision.
Composer 2 Technical Report
Cursor
A technical report on Composer 2, a specialized coding model trained through continued pretraining followed by large-scale reinforcement learning on long-horizon software-engineering tasks.
GLM-5: from Vibe Coding to Agentic Engineering
Z.ai
A GLM-5 technical report centered on agentic engineering, combining a more efficient long-context architecture with asynchronous reinforcement-learning infrastructure and agent RL for complex, long-horizon software tasks.
OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments
Tianbao Xie
OSWorld introduces a benchmark and environment for evaluating multimodal agents on open-ended tasks across real computer interfaces.
Eureka: Human-Level Reward Design via Coding Large Language Models
NVIDIA
Eureka uses a code-capable language model to propose and iteratively improve executable reward functions for reinforcement-learning tasks.
SWE-bench: Can Language Models Resolve Real-World GitHub Issues?
Princeton University
SWE-bench turns real GitHub issues and repository states into tasks for evaluating whether language models can produce working software patches.
WebArena: A Realistic Web Environment for Building Autonomous Agents
Carnegie Mellon University
WebArena provides self-hosted, realistic websites and benchmark tasks for agents that must navigate interfaces, maintain state, and complete multi-step goals.
Voyager: An Open-Ended Embodied Agent with Large Language Models
Guanzhi Wang
Voyager combines an automatic curriculum, an executable skill library, and iterative prompting to build an open-ended Minecraft agent without model parameter updates.
Mastering Diverse Domains through World Models
Google DeepMind
DreamerV3 presents a world-model-based reinforcement-learning algorithm designed to work across varied domains with a shared configuration.
MineDojo: Building Open-Ended Embodied Agents with Internet-Scale Knowledge
NVIDIA
MineDojo combines a Minecraft simulation suite, a broad task set, and internet-scale multimodal knowledge for research on generalist embodied agents.
Emergent Complexity and Zero-Shot Transfer via Unsupervised Environment Design
Michael Dennis
This work frames task-distribution design as an optimization problem and introduces PAIRED, where an environment generator uses regret to create solvable but challenging curricula.
Dota 2 with Large Scale Deep Reinforcement Learning
OpenAI
The OpenAI Five report documents a distributed, continually trained self-play system for a long-horizon, imperfect-information team game.