Topic
Group Relative Policy Optimization and related methods update a language-model policy using rewards from sampled response groups rather than a separately learned value critic. The family is attractive for reasoning post-training because it can simplify the RL stack, but performance still depends on sampling, normalization, clipping, KL control, and the quality of the reward signal.
Group-relative advantages
Policy gradients
KL control
Clipping
Latest research
Zhejiang University
A common evaluation compares ten agentic self-distillation methods with GRPO across Qwen3.5-2B, 4B, and 9B on four agent benchmarks. Across 132 training runs, gains depend strongly on the setting, final scores can conceal unstable learning, and additional training time often fails to improve performance.
Dalian Maritime University
A GSM8K study separates current-policy sample selection from progression between difficulty bands in GRPO. With Qwen2.5-1.5B-Instruct and LoRA, a staged curriculum reaching moderately harder problems improves test accuracy over persistent medium-difficulty selection, while a further difficulty increase reverses the gain across three seeds.
Alibaba Group
Zero-Sum Calibrated Policy Optimization (ZCPO) puts reference-policy drift into group-relative update coefficients instead of adding an independent KL loss. The paper analyzes seven potential KL–reward mismatches and reports improved math and long-context performance in its tested configurations.
Harbin Institute of Technology (Shenzhen)
GRPODropout selectively excludes high-probability positive-advantage rollouts from GRPO updates, then recenters retained advantages using old-policy probability weights. With the same generation budget, experiments on Qwen3 models report higher average benchmark accuracy and less entropy collapse than standard GRPO.
National University of Singapore
RISED uses shared behavioral rubrics to select rollout groups across agent environments and to condition an on-policy self-distillation teacher. It improves mean validation pass rate on two small-model backbones, while adding judge inference, teacher forward passes, and rollout oversampling.
University of Illinois Urbana-Champaign
DARS turns annotated task dependencies into signed step rewards: an unresolved mistake discounts dependent progress while independent work retains credit. Experiments cover household tasks, shopping, retrieval, and mathematics, with the clearest matched gain on a 1.5B ALFWorld agent.
Shuai Dong
On-policy distillation can prepare a student for later RL even when its immediate accuracy gain is small. Controlled downstream recipes show stronger reported final performance from OPD starting policies, and a distillation objective ranking that reverses after RL.
Juzheng Zhang
ActObs includes environment observations in the supervised loss on existing agent trajectories. The resulting initialization supports different exploration behavior during subsequent RL, with gains that depend on model size and evaluation sampling budget.
Together AI
Score centering removes an accumulating drift term caused by differences between rollout and training engines. Experiments test when this additive correction stabilizes quantized or stale-policy RL, and when it benefits from being combined with importance sampling.
Yan Yu
RetireOPD first trains a teacher with privileged task skills, then combines on-policy distillation with RL for a student that cannot access those skills. An adaptive rule retires the teacher once transfer stalls and the student reaches sufficient task competence.
Dialpad Inc.
A controlled study compares SFT with LoRA, GRPO, and their sequential combination across six Qwen3 model sizes. SFT leads most in-distribution tool-calling settings, while RL’s cross-dataset advantage is smaller and dataset mixing remains consistently useful.
Michael Noukhovitch
Never Give Up reallocates rollout compute toward hard problems by quickly filtering easy groups and repeatedly sampling unsolved ones. The study links this adaptive allocation to better learning signals on difficult mathematical and coding tasks.
Yang Li
RISE constructs a distillation teacher by extrapolating a model’s own RLVR training trajectory. Alternating reward-based updates with token-level distillation improves the tested reasoning and agent policies while reusing their sampled rollouts.
Sogang University
A GRPO replay method that reuses past rollout groups according to their remaining learning value and compatibility with the current policy. Experiments examine when replay can replace costly fresh environment interactions in reasoning and agent training.
New York University
An empirical study of how to combine on-policy distillation with RLVR. Training with distillation first and RL second produces strong logic-reasoning gains, while the advantage over the strongest alternatives is less decisive on math.
Jingxiao Yang
A work-in-progress study of long-horizon agent reinforcement learning that uses privileged process supervision to redistribute verified trajectory-level credit across executable actions.
Ziniu Li
An empirical decomposition of LLM reinforcement-learning batch scaling into sample-indexed learning behavior and systems throughput, with a decision rule for when a larger batch lowers wall-clock time-to-target.
Qiancheng Zhou
A controlled study of where reasoning diversity disappears during RLVR, distinguishing failure to initiate a solution family from failure to execute it after the first reasoning choice.
Maciej Besta
A compute-centric systematization of RL-style reasoning-model post-training, covering PPO and GRPO as well as intra-model, inter-model, synchronous, and asynchronous parallelism.
Zichen Liu
A controlled examination of R1-Zero-style reinforcement learning that separates robust training effects from claims that are sensitive to setup, reward design, and evaluation choices.
Weihao Zeng
An empirical study of reinforcement learning directly from open base models, focused on making zero-RL experiments easier to reproduce and compare across model families.
DAPO Team
DAPO presents an open reinforcement-learning system and a collection of training interventions aimed at improving large-scale reasoning optimization.
DeepSeek-AI
DeepSeekMath combines a math-focused data pipeline with supervised fine-tuning and reinforcement learning, and introduces Group Relative Policy Optimization.
Stanford University
DPO reformulates preference optimization as a direct classification-style objective, avoiding an explicit reward-model training stage and online policy optimization.
OpenAI
The InstructGPT work combines demonstrations, preference comparisons, reward modeling, and PPO to align language-model behavior with user instructions.
OpenAI
PPO introduces clipped and adaptive objectives intended to make policy-gradient optimization simpler to implement while limiting excessively large policy updates.