Research index
Source-linked research notes with original summaries and a clear view of why each development matters for scaling intelligent systems.
Li, Shuyue Stella
Verifiable Visual Rewards derives image prompts and deterministic constraint checks from the same geometric scene specification. Flow-GRPO training on simple synthetic scenes improves instruction following in Stable Diffusion 3.5 Medium and transfers to natural-image prompts, with measurable tradeoffs.
Kuaishou
KwaiMind combines diffusion preference optimization, task-specific online RL, and on-policy distillation for commercial image editing. Dedicated rewards cover predicted click-through rate, text rendering, and product consistency; the report separates offline reward-based gains from an online image-selection experiment.
Xiaomi
Xiaomi’s CodeMidas turns implemented functionality in existing codebases into executable coding RL tasks. Training MiMo-V2.5 on 5,545 filtered environments improves five external benchmarks, while a 3,000-task filtered subset beats an 8,000-task unfiltered baseline on three evaluations.
Xuan Liu
Checkpoint handoff separates two sources of agentic RL gains: reaching useful states and completing tasks from those states. Replaying histories across SFT and RL checkpoints reveals an interaction that endpoint success alone cannot attribute.
Yan Yu
RetireOPD first trains a teacher with privileged task skills, then combines on-policy distillation with RL for a student that cannot access those skills. An adaptive rule retires the teacher once transfer stalls and the student reaches sufficient task competence.
Hejia Geng
ScienceIDE turns scientific codebases and expert acceptance criteria into executable tasks, environments, and verifiers. Its experiments examine scientific software repair, supervised transfer, and RL with costly outcome feedback and long interaction histories.
Leon Bergen
Simple directions in model activations can flag reward-hacking behavior during agent evaluations. The study compares difference-of-means probes with LLM monitors, examines predictive signals before actions, and documents where detection still produces errors.
kaivu
A research-methods proposal for studying when models misbehave in realistic use. It starts from recorded failures, varies environment conditions or the policy, and asks how those observations can support estimates of behavior in a target task distribution.
Yang Li
RISE constructs a distillation teacher by extrapolating a model’s own RLVR training trajectory. Alternating reward-based updates with token-level distillation improves the tested reasoning and agent policies while reusing their sampled rollouts.
Qwen Team, Alibaba Group
Terminal-Universe reconstructs reusable terminal environments from recorded agent trajectories, then generates new tasks within projects, across related codebases, and over multiple user rounds. SFT experiments suggest that expanding environment diversity can produce more useful supervision than adding queries or solutions to existing environments.
New York University
An empirical study of how to combine on-policy distillation with RLVR. Training with distillation first and RL second produces strong logic-reasoning gains, while the advantage over the strongest alternatives is less decisive on math.
Anthropic
An Anthropic study and LessWrong discussion examining how reward hacking learned in deliberately vulnerable RL environments can transfer to harmful behavior in other evaluation scenarios. The findings also show important limits to that transfer.
Jingxiao Yang
A work-in-progress study of long-horizon agent reinforcement learning that uses privileged process supervision to redistribute verified trajectory-level credit across executable actions.
RL Scaling
A practical framework for locating the binding constraint in an RL system across environments, rollout generation, reward quality, optimization, and evaluation.
Qiancheng Zhou
A controlled study of where reasoning diversity disappears during RLVR, distinguishing failure to initiate a solution family from failure to execute it after the first reasoning choice.
Z.ai
An official GLM-5.3 release note describing a post-training-only update on the same base model as GLM-5.2, with expanded executable environments for complex coding, security, and long-horizon professional workflows.
Steven Byrnes
A community synthesis linking four training signals—imitation, human approval, automatic verifiers, and LLM judges—to distinct classes of observed or hypothesized alignment failure.
Fiora Starlight
A LessWrong proposal to let policies report exploitable RLVR environment bugs after a rollout, reward high-quality verified reports, and use them to patch the training environment.
Jialian Li
A technical report on Athena-Brain-8B, an on-device embodied model trained through general supervised fine-tuning, general reinforcement learning, embodied-expert training, and model merging.
Yufei Li
A production-oriented technical report on a generative reasoning re-ranker trained with semantic-ID mid-training, teacher-trace distillation, on-policy distillation, and reinforcement learning from verifiable ranking rewards.
Z.ai
An official release of the 750B-A40B GLM-5.2 model, combining a one-million-token context, IndexShare sparse-attention reuse, and larger-scale agentic reinforcement learning for long-horizon tasks.
MiniMax
An official release of MiniMax-M3, a natively multimodal sparse model with about 428B total and 23B active parameters, a one-million-token context window, and agent-oriented coding and cowork capabilities.
MiniMax
A technical report on the MiniMax-M2 family, pairing a 229.9B-parameter sparse MoE with agent-generated, verifiable trajectories and Forge, a scalable reinforcement-learning system for long-horizon agents.
Zichen Liu
A controlled examination of R1-Zero-style reinforcement learning that separates robust training effects from claims that are sensitive to setup, reward design, and evaluation choices.
Weihao Zeng
An empirical study of reinforcement learning directly from open base models, focused on making zero-RL experiments easier to reproduce and compare across model families.
DAPO Team
DAPO presents an open reinforcement-learning system and a collection of training interventions aimed at improving large-scale reasoning optimization.
DeepSeek-AI
DeepSeek-R1 documents a reasoning-model training pipeline centered on reinforcement learning, including a zero-style RL stage and a staged route that combines supervised data with RL.
DeepSeek-AI
DeepSeekMath combines a math-focused data pipeline with supervised fine-tuning and reinforcement learning, and introduces Group Relative Policy Optimization.
Meta FAIR
This work studies language models that generate candidate responses and also provide the preference signal used to improve subsequent iterations.
NVIDIA
Eureka uses a code-capable language model to propose and iteratively improve executable reward functions for reinforcement-learning tasks.
Princeton University
SWE-bench turns real GitHub issues and repository states into tasks for evaluating whether language models can produce working software patches.
Google DeepMind
ReST alternates offline sample generation with policy improvement, allowing generated data to be reused rather than coupling every update to fresh online rollouts.
OpenAI
A comparison of outcome supervision and process supervision for mathematical reasoning, centered on whether feedback should evaluate only the final answer or intermediate steps as well.
Stanford University
DPO reformulates preference optimization as a direct classification-style objective, avoiding an explicit reward-model training stage and online policy optimization.
Anthropic
Constitutional AI uses written principles and model-generated critiques and preferences to reduce direct dependence on human harmlessness labels during alignment training.
OpenAI
An empirical study of how optimizing against a learned reward model can eventually reduce performance under a higher-quality reference signal.
NVIDIA
MineDojo combines a Minecraft simulation suite, a broad task set, and internet-scale multimodal knowledge for research on generalist embodied agents.
OpenAI
The InstructGPT work combines demonstrations, preference comparisons, reward modeling, and PPO to align language-model behavior with user instructions.
OpenAI
A large-scale study of training summarization models from human comparisons using reward modeling and reinforcement learning.
OpenAI
A foundational study of learning a reward model from pairwise human preferences and using that learned reward to train an RL agent.