TL;DR
SURGE recombines updates from two checkpoints of the same RL run, retaining a leading spectral component from an accurate anchor and adding complementary donor updates. Across two 1.5B math histories and one 7B coding history, the constructed policies improve selected accuracy measures while generating fewer tokens than their anchors.
Why it matters
A saved RL trajectory may offer more than a choice of existing checkpoints. Reusing its update geometry is a concrete alternative to extending training, but its value depends on checkpoint selection, empirical calibration, and the cost of evaluating the resulting policy.
Key findings
- 01
SURGE subtracts the shared RL initialization before applying SVD to the anchor’s updates. It preserves a two-sided leading update block, fills its complement from the donor, and copies non-target parameters from the anchor. The output has the same architecture and runs as one policy.
- 02
The two math histories use DeepSeek-R1-Distill-Qwen-1.5B and OpenMath-Nemotron-1.5B, trained with the JustRL GRPO recipe on DAPO-Math-17k. Across nine equally weighted benchmarks, SURGE gains 2.08 and 1.78 percentage points over the respective anchors, with 5.3% and 12.8% fewer mean response tokens; gains are computed before rounding.
- 03
DeepSeek AIME24 accuracy is 54.17% versus the best of 19 measured native checkpoints at 50.83%. Math evaluation averages four sampled responses per problem under fixed decoding, rather than measuring success among four attempts. Nemotron improves on eight of nine benchmarks, with a 0.41-point decline on OlympiadBench.
- 04
For the OLMo-3.1-7B-RL-Zero-Code history trained from Olmo-3-1025-7B on Dolci-RL-Zero-Code-7B, HumanEval+ mean single-response pass rate is 83.7% versus 82.8% for the best of 19 measured native checkpoints. MBPP+ matches the anchor at reported precision, while generated tokens decline by about 10% on both coding benchmarks.
- 05
A retained-update-energy target of 0.95, chosen in the initial math study and fixed before OLMo, sets the fusion control from weights alone once the source pair is chosen. This avoids an accuracy sweep for that calibration step; source-policy evaluations still inform anchor and donor selection.
- 06
Measured warm-cache fusion takes 37.2 and 40.1 seconds for the two 1.5B models on one RTX A6000, including checkpoint read/write and spectral diagnostics. These timings exclude accuracy evaluation and do not establish 7B fusion latency.
Evidence & limits
The evidence covers three specific training histories and selected checkpoint pairs. It neither establishes a general scaling law nor guarantees improvement for arbitrary pairs, larger models, or other domains.
Four evaluation rounds measure sampling variability, not variation across independent RL training runs. The paper reports descriptive gains without claiming per-benchmark statistical significance; the small coding gains should be interpreted accordingly.
The retention target is empirical and was identified in the initial mathematical study. Weight-only fusion calibration does not make the entire source-selection and evaluation pipeline free of benchmark-informed choices.
Fusion and policy evaluation have separate costs. Comparisons with one RL step use reconstructed resource accounting that omits distributed overhead and other components, not measured end-to-end distributed training costs.
Best-native comparisons cover 19 measured checkpoints on DeepSeek AIME24 and OLMo HumanEval+ only. They do not show superiority over every checkpoint on every benchmark, and fewer generated tokens do not by themselves quantify serving latency savings.
Scaling dimensions
Models, methods & benchmarks
- Models
- DeepSeek-R1-Distill-Qwen-1.5B, OpenMath-Nemotron-1.5B, OLMo-3.1-7B-RL-Zero-Code
- Algorithms
- SURGE, Spectral checkpoint fusion, GRPO
- Benchmarks
- AIME 2024, AIME 2025, HMMT 2025, BRUMO 2025, CMIMC 2025, AMC 2023, MATH-500, Minerva, OlympiadBench, HumanEval+, MBPP+
Topics