TL;DR
Category-specific software-engineering experts alternate agentic RL with replay of their own successful trajectories, then transfer their gains into one student through label-routed on-policy distillation. The study tracks category regressions hidden by aggregate benchmark scores.
Why it matters
Scaling a mixed training pool can improve average performance while weakening particular task categories. This work tests whether specialization, policy-adaptive task selection, and multi-teacher integration can retain broader gains in a single deployed agent.
Key findings
- 01
All principal policies start from Qwen3.6-27B. Three expert routes cover service/data/security, user-facing applications, and systems/tooling/runtimes. Refresh–Repair–Expand reassesses task mastery, applies SFT to the policy’s own verified successes, and selects tasks for further RL.
- 02
The student generates its own trajectories and receives token-level signals from the category-routed teacher. A ReLU-gated teacher-versus-reference term adds extrapolation only where the teacher assigns higher probability; environment return does not enter this distillation-stage loss.
- 03
The final single model reaches 58.04% mean resolution on Pro-618 versus 52.64% for the base, 55.50% for Pooled RL, and 55.34% for Balanced RL. The paper reports a 5.39-percentage-point gain over base, calculated before rounding; results average three evaluation rounds.
- 04
On all 300 SWE-bench Multilingual tasks, the fixed final model reaches 59.00% versus 56.22% for base, 55.56% for Pooled RL, and 57.00% for Balanced RL. The authors report higher mean scores than both joint-RL baselines in each routed category, although recovery of the separate experts’ gains remains uneven.
Evidence & limits
Pro-618 excludes 113 of the 731 SWE-bench Pro instances and also informs checkpoint selection. Its results are not full SWE-bench Pro scores or a wholly untouched test-set estimate; SWE-bench Multilingual supplies the complementary fixed-model evaluation.
Expert training and teacher/reference forward passes add compute. The study does not establish a compute-matched efficiency advantage over one pooled policy.
The experiments evaluate the complete framework and stage-wise outcomes rather than isolating every optimization, routing, or replay component. Three evaluation rounds measure evaluation variation, not independent training-run robustness.
No external model supplies solution trajectories or action targets during expert training and integration, but Qwen3.7-max labels the task pools. The pipeline should not be described as entirely free of external-model assistance.
Hard category routing simplifies overlapping tasks. Executable tests and sanitization cannot eliminate pretraining memorization, specification defects, or every verifier exploit.
Scaling dimensions
Models, methods & benchmarks
- Models
- Qwen3.6-27B
- Algorithms
- Agentic-miniRL, Refresh–Repair–Expand, Multi-teacher on-policy distillation
- Benchmarks
- Pro-618, SWE-bench Multilingual
Topics