TL;DR
A GSM8K study separates current-policy sample selection from progression between difficulty bands in GRPO. With Qwen2.5-1.5B-Instruct and LoRA, a staged curriculum reaching moderately harder problems improves test accuracy over persistent medium-difficulty selection, while a further difficulty increase reverses the gain across three seeds.
Why it matters
Mixed-success response groups can provide useful GRPO learning signals without defining the best curriculum for final performance. The study suggests monitoring both which problems are currently learnable and whether moving to a harder band still helps, rather than assuming that more difficult training is always better.
Key findings
- 01
The experiments use Qwen2.5-1.5B-Instruct with LoRA on GSM8K. Offline difficulty tracking samples 32 responses per problem; online curriculum selection uses eight. Correctness determines difficulty gates, while training also includes an auxiliary length reward weighted 0.2 with a preferred range of 60–650 tokens.
- 02
The fixed-medium curriculum continually reselects problems with three to five correct responses out of eight. The staged procedure progresses through easier bands to a moderately hard band with two to four correct responses, then a harder band with one to three. These success-rate intervals overlap and are relative to the current policy.
- 03
Across seeds 42, 2026, and 100, the selected moderately hard checkpoint achieves 65.91% ± 0.77 percentage points GSM8K test accuracy versus 64.49% ± 0.24 for fixed-medium training: a mean gain of 1.42 percentage points. The reported uncertainty is standard deviation across three seeds, not a confidence interval.
- 04
After 100 additional effective updates in the harder band, mean test accuracy drops to 64.05% ± 0.53 percentage points, 1.86 points below the selected moderately hard checkpoint. This decline occurs in each of the three reported seeds.
- 05
The staged procedure reserves five disjoint diagnostic subsets of 128 training problems each, leaving 6,833 training candidates. It evaluates the active diagnostic subset every 50 effective updates and uses a fixed patience rule for stage transitions and checkpoint retention; the test set does not drive those decisions.
- 06
In a separate difficulty-tracking run over 1,495 problems, initial-to-final difficulty ranks remain correlated (Spearman 0.8092), yet only 20.74% of initially medium-band problems remain in that band. A stable overall ranking therefore coexists with substantial local membership change.
- 07
The static easy-to-hard curriculum remains competitive at 65.22% ± 0.22 percentage points best-observed test accuracy. The selected staged result is 0.69 points higher on average; stale difficulty estimates alone do not establish that a static curriculum is ineffective.
Evidence & limits
The study covers one 1.5B model with LoRA and one benchmark. It does not establish an optimal difficulty schedule for larger models, coding, or multi-turn agents.
Standard GRPO, static curriculum, and fixed-medium baselines report retrospective best-observed test accuracy along completed trajectories. Staged checkpoints are selected using held-out diagnostic subsets from the training split. These are different selection protocols and should not be presented as identical final-checkpoint comparisons.
The staged procedure has a smaller training candidate pool, diagnostic evaluations, and a predefined transition rule. The authors describe the comparisons as complete-procedure comparisons, not strict single-factor tests isolating difficulty progression.
Only eight responses estimate online solvability, so sampling noise can move problems between overlapping bands. Offline direction-reversal counts are sensitive to the chosen minimum-change threshold.
An effective update counts an optimizer step with at least one accepted sample. It does not count all sampling or diagnostic work, and the reported accuracy gains do not establish a wall-clock or total-compute saving.
The harder-band result follows 100 extra updates after a selected earlier checkpoint. It supports a non-monotonic pattern in the tested procedure, not a universal causal claim that hard examples damage learning.
Scaling dimensions
Models, methods & benchmarks
- Models
- Qwen2.5-1.5B-Instruct
- Algorithms
- GRPO, LoRA, Online difficulty selection, Staged curriculum
- Benchmarks
- GSM8K
Topics