TL;DR
Never Give Up reallocates rollout compute toward hard problems by quickly filtering easy groups and repeatedly sampling unsolved ones. The study links this adaptive allocation to better learning signals on difficult mathematical and coding tasks.
Why it matters
More rollout compute is useful only when it produces informative updates. Adaptive sampling offers a way to balance easy and hard tasks as the policy changes, rather than choosing one fixed group size for every prompt.
Key findings
- 01
Standard RL improves easier problems more than harder ones in the studied settings. For unfamiliar coding tasks, difficulty must be assessed after initial adaptation to the prompt and harness to reveal this pattern.
- 02
NGU samples a small group, quickly filters all-correct groups, and can keep sampling all-fail groups until a success appears. A probabilistic stopping rule bounds expected sampling rather than literally continuing forever; asynchronous execution refills available slots.
- 03
On a 10,000-problem Deepscaler subset, Qwen3-4B-Base runs are matched at approximately 120 H100-hours with three seeds per setting. Evaluation on AIME 2025 and BRUMO November 2025 shows a better easy/hard performance trade-off than fixed group sizes and the tested fixed-difficulty curriculum.
- 04
On Manufactoria coding, NGU improves learning on hard test cases and enables fully solved problems where the per-test-reward GRPO baseline plateaus. The method filters overly stale negative completions while retaining their reward counts for advantage construction.
Evidence & limits
The method relies on a useful mixture of easy and hard problems. If almost all problems are hard, repeated small sampling groups may be less efficient than a larger group from the start.
Asynchronous execution is central to reallocating compute, but repeated sampling also creates stale, off-policy completions. Benefits depend on controlling this staleness; retaining older negatives can hurt learning.
Evidence covers the studied mathematical and coding tasks, not a general demonstration for multi-step agent environments. The compute-matched Deepscaler comparison should not be generalized into a universal speedup.
Scaling dimensions
Models, methods & benchmarks
- Models
- Qwen3-4B-Base, Qwen3-4B-Instruct
- Algorithms
- Never Give Up, GRPO, Adaptive sampling, Asynchronous RL
- Benchmarks
- Deepscaler, AIME 2025, BRUMO November 2025, Manufactoria
Topics