Boosting LLM Exploration via Weak-Model Guidance in RLVR
Xingyu Shen
A method for preserving reasoning diversity during RLVR by conditioning the target model on partial trajectories produced by a smaller, weaker model.
Topic
Post-training
Post-training turns a pretrained model into a useful system through supervised fine-tuning, preference optimization, reinforcement learning, distillation, and targeted data generation. The choice of method should follow the desired behavior, available feedback, and evaluation setup rather than a preference for one algorithm.
RLHF
RLAIF
Preference optimization
Distillation
Latest research
Performance Foundations of Parallel & Distributed Reasoning Language Models
Maciej Besta
A compute-centric systematization of RL-style reasoning-model post-training, covering PPO and GRPO as well as intra-model, inter-model, synchronous, and asynchronous parallelism.
GLM-5.3: Frontier Coding with Emergent Cyber Capabilities
Z.ai
An official GLM-5.3 release note describing a post-training-only update on the same base model as GLM-5.2, with expanded executable environments for complex coding, security, and long-horizon professional workflows.
Kimi K3: Open Frontier Intelligence
Moonshot AI
A technical report on Kimi K3, a 2.8T-parameter sparse multimodal model with 104B active parameters and a one-million-token context window, post-trained with reinforcement learning across general, agentic, coding, and reasoning domains.
LLMs are (still) mostly powered by imitative learning, not RL
Steven Byrnes
A community argument that RLVR is operationally important but that most language-model knowledge and available reasoning strategies still originate in pretraining and supervised imitation.
Athena-Brain Technical Report: An Efficient Robot Brain for General Intelligence and Embodied Interaction
Jialian Li
A technical report on Athena-Brain-8B, an on-device embodied model trained through general supervised fine-tuning, general reinforcement learning, embodied-expert training, and model merging.
GR2 Technical Report
Yufei Li
A production-oriented technical report on a generative reasoning re-ranker trained with semantic-ID mid-training, teacher-trace distillation, on-policy distillation, and reinforcement learning from verifiable ranking rewards.
Sakana Fugu Technical Report
Sakana AI
A technical report on Fugu and Fugu-Ultra, language-model orchestrators trained to construct query-adaptive scaffolds for teams of heterogeneous LLM agents.
GLM-5.2: Built for Long-Horizon Tasks
Z.ai
An official release of the 750B-A40B GLM-5.2 model, combining a one-million-token context, IndexShare sparse-attention reuse, and larger-scale agentic reinforcement learning for long-horizon tasks.
Ling and Ring 2.6 Technical Report: Efficient and Instant Agentic Intelligence at Trillion-Parameter Scale
Ang Li
A model-family report on Ling-2.6 and Ring-2.6, combining architectural migration, long-context efficiency work, token-efficient reasoning objectives, and asynchronous agent reinforcement learning at trillion-parameter scale.
MiniMax-M3: Native Multimodal Intelligence at 1M Context
MiniMax
An official release of MiniMax-M3, a natively multimodal sparse model with about 428B total and 23B active parameters, a one-million-token context window, and agent-oriented coding and cowork capabilities.
Some Interesting Papers on RLVR
CarolusRenniusVitellius
A concise reading map of weight-level and behavioral evidence on whether RLVR mainly reweights existing capabilities or creates new reasoning mechanisms.
The MiniMax-M2 Series: Mini Activations Unleashing Max Real-World Intelligence
MiniMax
A technical report on the MiniMax-M2 family, pairing a 229.9B-parameter sparse MoE with agent-generated, verifiable trajectories and Forge, a scalable reinforcement-learning system for long-horizon agents.
GLM-5.1: Towards Long-Horizon Tasks
Z.ai
An official GLM-5.1 model update focused on keeping an agent productive across longer coding and engineering runs through repeated execution, inspection, diagnosis, and strategy revision.
Composer 2 Technical Report
Cursor
A technical report on Composer 2, a specialized coding model trained through continued pretraining followed by large-scale reinforcement learning on long-horizon software-engineering tasks.
GLM-5: from Vibe Coding to Agentic Engineering
Z.ai
A GLM-5 technical report centered on agentic engineering, combining a more efficient long-context architecture with asynchronous reinforcement-learning infrastructure and agent RL for complex, long-horizon software tasks.
DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
DeepSeek-AI
DeepSeek-R1 documents a reasoning-model training pipeline centered on reinforcement learning, including a zero-style RL stage and a staged route that combines supervised data with RL.
Kimi k1.5: Scaling Reinforcement Learning with LLMs
Kimi Team
A technical report on scaling multimodal language-model reinforcement learning with long-context training, policy optimization, and data-selection strategies.
DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models
DeepSeek-AI
DeepSeekMath combines a math-focused data pipeline with supervised fine-tuning and reinforcement learning, and introduces Group Relative Policy Optimization.
Self-Rewarding Language Models
Meta FAIR
This work studies language models that generate candidate responses and also provide the preference signal used to improve subsequent iterations.
Reinforced Self-Training (ReST) for Language Modeling
Google DeepMind
ReST alternates offline sample generation with policy improvement, allowing generated data to be reused rather than coupling every update to fresh online rollouts.
Direct Preference Optimization: Your Language Model is Secretly a Reward Model
Stanford University
DPO reformulates preference optimization as a direct classification-style objective, avoiding an explicit reward-model training stage and online policy optimization.
Constitutional AI: Harmlessness from AI Feedback
Anthropic
Constitutional AI uses written principles and model-generated critiques and preferences to reduce direct dependence on human harmlessness labels during alignment training.
Training Language Models to Follow Instructions with Human Feedback
OpenAI
The InstructGPT work combines demonstrations, preference comparisons, reward modeling, and PPO to align language-model behavior with user instructions.
Learning to Summarize from Human Feedback
OpenAI
A large-scale study of training summarization models from human comparisons using reward modeling and reinforcement learning.