TL;DR
A production-oriented technical report on a generative reasoning re-ranker trained with semantic-ID mid-training, teacher-trace distillation, on-policy distillation, and reinforcement learning from verifiable ranking rewards.
Why it matters
GR2 moves RLVR beyond math and code into an industrial recommendation funnel. It also documents an operational scaling failure: models learned to preserve incoming order or exploit position bias when the reward allowed it.
Key findings
- 01
The system combines distilled reasoning traces with RL on verifiable rewards designed specifically for re-ranking.
- 02
The report introduces On-Policy Distillation as a scalable alternative after supervised fine-tuning reportedly collapsed at industrial scale.
- 03
The authors report online ranking gains over legacy baselines and argue that conditional rewards are necessary to limit order-preservation and position-bias reward hacks.
Scaling dimensions
Models, methods & benchmarks
- Models
- GR2
- Algorithms
- RLVR, On-Policy Distillation, Reasoning distillation
- Benchmarks
- Industrial re-ranking traffic
Topics