TL;DR
A controlled study compares SFT with LoRA, GRPO, and their sequential combination across six Qwen3 model sizes. SFT leads most in-distribution tool-calling settings, while RL’s cross-dataset advantage is smaller and dataset mixing remains consistently useful.
Why it matters
Choosing RL should depend on the target distribution and training budget. This study adds a useful counterweight to benchmark-only claims by separating model scale, adaptation method, and data composition for tool calling.
Key findings
- 01
Experiments span Qwen3 models from 0.6B to 32B and compare SFT-LoRA, GRPO, and SFT followed by GRPO under both in-distribution evaluation and cross-dataset transfer.
- 02
SFT-LoRA is strongest in 15 of 18 in-distribution settings. GRPO wins 29 of 54 cross-dataset settings, but its average margin over SFT is under one point; the sequential SFT-to-GRPO recipe is rarely the strongest.
- 03
Mixing datasets gives consistently strong transfer while staying close to specialized in-distribution training across methods. Data composition remains important independently of the chosen optimizer.
- 04
A comparison at 4B finds LoRA more effective than full-parameter fine-tuning, consistent with better preservation of pretrained agentic behavior in that tested setting.
Evidence & limits
The comparison is not compute-normalized: GRPO consumes substantially more compute per training example than SFT. The results do not directly determine a compute-optimal training recipe.
All six scale points belong to Qwen3. Replication in another model family is needed before treating the rankings as general.
LoRA versus full-parameter fine-tuning is tested only at 4B. Tool-calling results also do not establish that SFT dominates RL in long-horizon interactive environments.
Scaling dimensions
Models, methods & benchmarks
- Models
- Qwen3-0.6B, Qwen3-1.7B, Qwen3-4B, Qwen3-8B, Qwen3-14B, Qwen3-32B
- Algorithms
- Supervised Fine-Tuning, LoRA, GRPO
- Benchmarks
- xLAM, ToolACE, Glaive-FC-v2
Topics