Topic
Group Relative Policy Optimization and related methods update a language-model policy using rewards from sampled response groups rather than a separately learned value critic. The family is attractive for reasoning post-training because it can simplify the RL stack, but performance still depends on sampling, normalization, clipping, KL control, and the quality of the reward signal.
Group-relative advantages
Policy gradients
KL control
Clipping
Latest research
Jingxiao Yang
A work-in-progress study of long-horizon agent reinforcement learning that uses privileged process supervision to redistribute verified trajectory-level credit across executable actions.
Ziniu Li
An empirical decomposition of LLM reinforcement-learning batch scaling into sample-indexed learning behavior and systems throughput, with a decision rule for when a larger batch lowers wall-clock time-to-target.
Qiancheng Zhou
A controlled study of where reasoning diversity disappears during RLVR, distinguishing failure to initiate a solution family from failure to execute it after the first reasoning choice.
Maciej Besta
A compute-centric systematization of RL-style reasoning-model post-training, covering PPO and GRPO as well as intra-model, inter-model, synchronous, and asynchronous parallelism.
Zichen Liu
A controlled examination of R1-Zero-style reinforcement learning that separates robust training effects from claims that are sensitive to setup, reward design, and evaluation choices.
Weihao Zeng
An empirical study of reinforcement learning directly from open base models, focused on making zero-RL experiments easier to reproduce and compare across model families.
DAPO Team
DAPO presents an open reinforcement-learning system and a collection of training interventions aimed at improving large-scale reasoning optimization.
DeepSeek-AI
DeepSeekMath combines a math-focused data pipeline with supervised fine-tuning and reinforcement learning, and introduces Group Relative Policy Optimization.
Stanford University
DPO reformulates preference optimization as a direct classification-style objective, avoiding an explicit reward-model training stage and online policy optimization.
OpenAI
The InstructGPT work combines demonstrations, preference comparisons, reward modeling, and PPO to align language-model behavior with user instructions.
OpenAI
PPO introduces clipped and adaptive objectives intended to make policy-gradient optimization simpler to implement while limiting excessively large policy updates.