Topic
Agentic RL applies reinforcement learning to systems that plan, use tools, interact with software, and act over long horizons. The hard problems extend beyond the base model: state, memory, environment resets, credit assignment, reliable verifiers, and safe exploration all become part of the training system.
Tool use
Long-horizon credit
Memory
Workflow environments
Latest research
Zhejiang University
KnowGPO groups search-agent steps by accumulated task-relevant knowledge and rewards newly acquired facts. Controlled Qwen3-8B comparisons show gains over GRPO on four evaluation splits, but the method requires reference answers, successful training trajectories, and an external LLM verifier. Several narrative claims in the first version disagree with its tables.
Tsinghua University
ActionOPD distills black-box agent teachers by rewarding semantic agreement between sampled student and teacher actions. It needs teacher generation rather than token probabilities or aligned tokenizers. Experiments report cross-benchmark transfer, alongside mixed success-versus-reliability results against token-level OPD.
Zhejiang University
A common evaluation compares ten agentic self-distillation methods with GRPO across Qwen3.5-2B, 4B, and 9B on four agent benchmarks. Across 132 training runs, gains depend strongly on the setting, final scores can conceal unstable learning, and additional training time often fails to improve performance.
Tsinghua University
BranchOPD selects decision points using student uncertainty and teacher–student disagreement, samples student continuations from shared prefixes, and distills teacher feedback on the resulting branches. Protocol-boundary repair (PBR) separately addresses repetition and malformed response boundaries. Experiments cover mathematical reasoning and three agent benchmarks.
Xiaomi
Xiaomi’s MiMo-V2.6 report details large-scale mixed-task agentic RL: long-horizon batches, diverse environments and harnesses, and agentic grading beyond binary test rewards. It reports training-cost curves, router-stability experiments, infrastructure failures, and a subsequent multi-teacher distillation stage.
NVIDIA
NeMo-DCR transfers compressed weight changes between training and rollout clusters while reconstructing the same parameter and buffer bits as a dense refit. Cross-region benchmarks report large synchronization speedups, with separate small-scale training experiments testing recovery from interrupted updates.
Institute of Computing Technology, Chinese Academy of Sciences
VenusRL separates training, rollout generation, and tool environments, prioritizing trajectory groups that can unblock the next training batch. It combines priority-aware scheduling and KV-cache management with denser sandbox placement, reporting workload-dependent throughput gains on long-horizon software-agent training.
National University of Singapore
RISED uses shared behavioral rubrics to select rollout groups across agent environments and to condition an on-policy self-distillation teacher. It improves mean validation pass rate on two small-model backbones, while adding judge inference, teacher forward passes, and rollout oversampling.
University of Illinois Urbana-Champaign
DARS turns annotated task dependencies into signed step rewards: an unresolved mistake discounts dependent progress while independent work retains credit. Experiments cover household tasks, shopping, retrieval, and mathematics, with the clearest matched gain on a 1.5B ALFWorld agent.
WeChat AI
WeEnv manages agentic RL environments across packaging, initialization, and resource provisioning. Composable layers and on-demand fetching reduce environment startup costs, while elastic CPU and memory quotas adapt to task demand.
Jie Zhao
Category-specific software-engineering experts alternate agentic RL with replay of their own successful trajectories, then transfer their gains into one student through label-routed on-policy distillation. The study tracks category regressions hidden by aggregate benchmark scores.
DeepSeek
DeepSeek Elastic Compute separates stateful agent execution from preemptible GPU training. Its sandbox platform combines on-demand image loading, composable filesystem layers, memory reclamation, and CPU scheduling to support large bursts of long-running agent workloads.
Xiaomi
Xiaomi’s CodeMidas turns implemented functionality in existing codebases into executable coding RL tasks. Training MiMo-V2.5 on 5,545 filtered environments improves five external benchmarks, while a 3,000-task filtered subset beats an 8,000-task unfiltered baseline on three evaluations.
Xuan Liu
Checkpoint handoff separates two sources of agentic RL gains: reaching useful states and completing tasks from those states. Replaying histories across SFT and RL checkpoints reveals an interaction that endpoint success alone cannot attribute.
Juzheng Zhang
ActObs includes environment observations in the supervised loss on existing agent trajectories. The resulting initialization supports different exploration behavior during subsequent RL, with gains that depend on model size and evaluation sampling budget.
Yan Yu
RetireOPD first trains a teacher with privileged task skills, then combines on-policy distillation with RL for a student that cannot access those skills. An adaptive rule retires the teacher once transfer stalls and the student reaches sufficient task competence.
Hejia Geng
ScienceIDE turns scientific codebases and expert acceptance criteria into executable tasks, environments, and verifiers. Its experiments examine scientific software repair, supervised transfer, and RL with costly outcome feedback and long interaction histories.
Leon Bergen
Simple directions in model activations can flag reward-hacking behavior during agent evaluations. The study compares difference-of-means probes with LLM monitors, examines predictive signals before actions, and documents where detection still produces errors.
Dialpad Inc.
A controlled study compares SFT with LoRA, GRPO, and their sequential combination across six Qwen3 model sizes. SFT leads most in-distribution tool-calling settings, while RL’s cross-dataset advantage is smaller and dataset mixing remains consistently useful.
RadixArk
Miles v0.1 describes a full-stack post-training system spanning rollout generation, optimization, and weight synchronization. Its reference run demonstrates asynchronous terminal-agent RL with a 744B-parameter MoE model and makes the system’s supported configurations explicit.
kaivu
A research-methods proposal for studying when models misbehave in realistic use. It starts from recorded failures, varies environment conditions or the policy, and asks how those observations can support estimates of behavior in a target task distribution.
Shanghai Jiao Tong University
CUA-Universe turns desktop applications into shared-state GUI and CLI environments, then generates tasks and training trajectories. Supervised fine-tuning teaches a 9B model to coordinate the two interfaces with higher task scores and lower interaction costs.
Sogang University
A GRPO replay method that reuses past rollout groups according to their remaining learning value and compatibility with the current policy. Experiments examine when replay can replace costly fresh environment interactions in reasoning and agent training.
Anthropic
An Anthropic study and LessWrong discussion examining how reward hacking learned in deliberately vulnerable RL environments can transfer to harmful behavior in other evaluation scenarios. The findings also show important limits to that transfer.
Jingxiao Yang
A work-in-progress study of long-horizon agent reinforcement learning that uses privileged process supervision to redistribute verified trajectory-level credit across executable actions.
Z.ai
An official GLM-5.3 release note describing a post-training-only update on the same base model as GLM-5.2, with expanded executable environments for complex coding, security, and long-horizon professional workflows.
Steven Byrnes
A community synthesis linking four training signals—imitation, human approval, automatic verifiers, and LLM judges—to distinct classes of observed or hypothesized alignment failure.
Fiora Starlight
A LessWrong proposal to let policies report exploitable RLVR environment bugs after a rollout, reward high-quality verified reports, and use them to patch the training environment.
Moonshot AI
A technical report on Kimi K3, a 2.8T-parameter sparse multimodal model with 104B active parameters and a one-million-token context window, post-trained with reinforcement learning across general, agentic, coding, and reasoning domains.
Jialian Li
A technical report on Athena-Brain-8B, an on-device embodied model trained through general supervised fine-tuning, general reinforcement learning, embodied-expert training, and model merging.
Sakana AI
A technical report on Fugu and Fugu-Ultra, language-model orchestrators trained to construct query-adaptive scaffolds for teams of heterogeneous LLM agents.
Z.ai
An official release of the 750B-A40B GLM-5.2 model, combining a one-million-token context, IndexShare sparse-attention reuse, and larger-scale agentic reinforcement learning for long-horizon tasks.
Ang Li
A model-family report on Ling-2.6 and Ring-2.6, combining architectural migration, long-context efficiency work, token-efficient reasoning objectives, and asynchronous agent reinforcement learning at trillion-parameter scale.
MiniMax
An official release of MiniMax-M3, a natively multimodal sparse model with about 428B total and 23B active parameters, a one-million-token context window, and agent-oriented coding and cowork capabilities.
MiniMax
A technical report on the MiniMax-M2 family, pairing a 229.9B-parameter sparse MoE with agent-generated, verifiable trajectories and Forge, a scalable reinforcement-learning system for long-horizon agents.
Z.ai
An official GLM-5.1 model update focused on keeping an agent productive across longer coding and engineering runs through repeated execution, inspection, diagnosis, and strategy revision.
Cursor
A technical report on Composer 2, a specialized coding model trained through continued pretraining followed by large-scale reinforcement learning on long-horizon software-engineering tasks.
Z.ai
A GLM-5 technical report centered on agentic engineering, combining a more efficient long-context architecture with asynchronous reinforcement-learning infrastructure and agent RL for complex, long-horizon software tasks.
Tianbao Xie
OSWorld introduces a benchmark and environment for evaluating multimodal agents on open-ended tasks across real computer interfaces.
NVIDIA
Eureka uses a code-capable language model to propose and iteratively improve executable reward functions for reinforcement-learning tasks.
Princeton University
SWE-bench turns real GitHub issues and repository states into tasks for evaluating whether language models can produce working software patches.
Carnegie Mellon University
WebArena provides self-hosted, realistic websites and benchmark tasks for agents that must navigate interfaces, maintain state, and complete multi-step goals.
Guanzhi Wang
Voyager combines an automatic curriculum, an executable skill library, and iterative prompting to build an open-ended Minecraft agent without model parameter updates.
NVIDIA
MineDojo combines a Minecraft simulation suite, a broad task set, and internet-scale multimodal knowledge for research on generalist embodied agents.