TL;DR
A method for preserving reasoning diversity during RLVR by conditioning the target model on partial trajectories produced by a smaller, weaker model.
Why it matters
RLVR often improves pass@1 while entropy and large-k coverage decline. The paper tests whether cross-model prefixes can create useful exploration pressure without an additional supervised-training stage or a more elaborate reward.
Key findings
- 01
The method uses unfamiliar weak-model prefixes to interrupt overconfident generation and expose the policy to different reasoning paths.
- 02
Across the reported mathematical benchmarks, gains increase as k grows, which the authors interpret as expanded reasoning coverage.
- 03
The approach is designed to mitigate entropy collapse without additional SFT, reward redesign, or complex prompting.
Scaling dimensions
Models, methods & benchmarks
- Algorithms
- RLVR, Weak-model guidance
Topics