Research index
Source-linked research notes with original summaries and a clear view of why each development matters for scaling intelligent systems.
Juzheng Zhang
ActObs includes environment observations in the supervised loss on existing agent trajectories. The resulting initialization supports different exploration behavior during subsequent RL, with gains that depend on model size and evaluation sampling budget.
Yuxiao Yang
On-policy distillation can suppress a student’s stopping behavior when teacher and student favor different termination tokens. Grouping operationally equivalent EOS tokens at the probability level mitigates this mechanism, while a separate late-stage source of length inflation remains unresolved.
Together AI
Score centering removes an accumulating drift term caused by differences between rollout and training engines. Experiments test when this additive correction stabilizes quantized or stale-policy RL, and when it benefits from being combined with importance sampling.
AI Singapore
SEA-LION v4.8 adapts Nemotron models to Southeast Asian languages through continued pretraining, supervised fine-tuning, online on-policy distillation, and model merging. The report connects regional language performance to both data coverage and tokenizer constraints.
Michael Noukhovitch
Never Give Up reallocates rollout compute toward hard problems by quickly filtering easy groups and repeatedly sampling unsolved ones. The study links this adaptive allocation to better learning signals on difficult mathematical and coding tasks.
RadixArk
Miles v0.1 describes a full-stack post-training system spanning rollout generation, optimization, and weight synchronization. Its reference run demonstrates asynchronous terminal-agent RL with a 744B-parameter MoE model and makes the system’s supported configurations explicit.
Yang Li
RISE constructs a distillation teacher by extrapolating a model’s own RLVR training trajectory. Alternating reward-based updates with token-level distillation improves the tested reasoning and agent policies while reusing their sampled rollouts.
Tsinghua University
An empirical study of how prompt difficulty and reasoning length affect on-policy distillation. Eight selected hard math problems approach the performance of a 17K-problem baseline, while repeated rollout generation and dense teacher supervision remain part of training.
Sogang University
A GRPO replay method that reuses past rollout groups according to their remaining learning value and compatibility with the current policy. Experiments examine when replay can replace costly fresh environment interactions in reasoning and agent training.
New York University
An empirical study of how to combine on-policy distillation with RLVR. Training with distillation first and RL second produces strong logic-reasoning gains, while the advantage over the strongest alternatives is less decisive on math.
Jingxiao Yang
A work-in-progress study of long-horizon agent reinforcement learning that uses privileged process supervision to redistribute verified trajectory-level credit across executable actions.
RL Scaling
A practical framework for locating the binding constraint in an RL system across environments, rollout generation, reward quality, optimization, and evaluation.
Ziniu Li
An empirical decomposition of LLM reinforcement-learning batch scaling into sample-indexed learning behavior and systems throughput, with a decision rule for when a larger batch lowers wall-clock time-to-target.
Qiancheng Zhou
A controlled study of where reasoning diversity disappears during RLVR, distinguishing failure to initiate a solution family from failure to execute it after the first reasoning choice.
Xingyu Shen
A method for preserving reasoning diversity during RLVR by conditioning the target model on partial trajectories produced by a smaller, weaker model.
Maciej Besta
A compute-centric systematization of RL-style reasoning-model post-training, covering PPO and GRPO as well as intra-model, inter-model, synchronous, and asynchronous parallelism.
Z.ai
An official GLM-5.3 release note describing a post-training-only update on the same base model as GLM-5.2, with expanded executable environments for complex coding, security, and long-horizon professional workflows.
Moonshot AI
A technical report on Kimi K3, a 2.8T-parameter sparse multimodal model with 104B active parameters and a one-million-token context window, post-trained with reinforcement learning across general, agentic, coding, and reasoning domains.
Steven Byrnes
A community argument that RLVR is operationally important but that most language-model knowledge and available reasoning strategies still originate in pretraining and supervised imitation.
Sakana AI
A technical report on Fugu and Fugu-Ultra, language-model orchestrators trained to construct query-adaptive scaffolds for teams of heterogeneous LLM agents.
Z.ai
An official release of the 750B-A40B GLM-5.2 model, combining a one-million-token context, IndexShare sparse-attention reuse, and larger-scale agentic reinforcement learning for long-horizon tasks.
Ang Li
A model-family report on Ling-2.6 and Ring-2.6, combining architectural migration, long-context efficiency work, token-efficient reasoning objectives, and asynchronous agent reinforcement learning at trillion-parameter scale.
MiniMax
An official release of MiniMax-M3, a natively multimodal sparse model with about 428B total and 23B active parameters, a one-million-token context window, and agent-oriented coding and cowork capabilities.
CarolusRenniusVitellius
A concise reading map of weight-level and behavioral evidence on whether RLVR mainly reweights existing capabilities or creates new reasoning mechanisms.
MiniMax
A technical report on the MiniMax-M2 family, pairing a 229.9B-parameter sparse MoE with agent-generated, verifiable trajectories and Forge, a scalable reinforcement-learning system for long-horizon agents.
Z.ai
An official GLM-5.1 model update focused on keeping an agent productive across longer coding and engineering runs through repeated execution, inspection, diagnosis, and strategy revision.
Cursor
A technical report on Composer 2, a specialized coding model trained through continued pretraining followed by large-scale reinforcement learning on long-horizon software-engineering tasks.
Z.ai
A GLM-5 technical report centered on agentic engineering, combining a more efficient long-context architecture with asynchronous reinforcement-learning infrastructure and agent RL for complex, long-horizon software tasks.
Zichen Liu
A controlled examination of R1-Zero-style reinforcement learning that separates robust training effects from claims that are sensitive to setup, reward design, and evaluation choices.
Weihao Zeng
An empirical study of reinforcement learning directly from open base models, focused on making zero-RL experiments easier to reproduce and compare across model families.
DAPO Team
DAPO presents an open reinforcement-learning system and a collection of training interventions aimed at improving large-scale reasoning optimization.
DeepSeek-AI
DeepSeek-R1 documents a reasoning-model training pipeline centered on reinforcement learning, including a zero-style RL stage and a staged route that combines supervised data with RL.
Kimi Team
A technical report on scaling multimodal language-model reinforcement learning with long-context training, policy optimization, and data-selection strategies.
Google DeepMind
ReST alternates offline sample generation with policy improvement, allowing generated data to be reused rather than coupling every update to fresh online rollouts.
OpenAI
The OpenAI Five report documents a distributed, continually trained self-play system for a long-horizon, imperfect-information team game.
OpenAI
PPO introduces clipped and adaptive objectives intended to make policy-gradient optimization simpler to implement while limiting excessively large policy updates.