TL;DR
Harness Scaling trains agent policies across structured variations of their execution harness instead of one fixed setup. A balanced 18-configuration subset improves held-out combinations and transfer to external harnesses in two Qwen models with GRPO and PPO. More configurations are not uniformly better under a fixed update budget.
Why it matters
Agent performance depends on the interface around the model as well as task content. Varying prompts, tools, feedback, context management and workflow during RL offers a way to reduce dependence on one runtime. The results make harness selection and task assignment concrete experimental variables rather than assuming a larger configuration pool is always beneficial.
Key findings
- 01
The paper factorizes the harness into five components: system prompt, tool interface, observation, context management and workflow. Three realizations per component yield 243 combinations. A strength-2 orthogonal design selects 18 training configurations with balanced component frequencies and pairwise combinations.
- 02
Each task is assigned a uniformly sampled anchor and a nearby configuration under Hamming distance, used across two successive updates. Training covers 13.5K ClawGym-Task workspace tasks for 300 optimization steps. GRPO uses 16 tasks with eight rollouts each; PPO uses 128 tasks with one rollout each.
- 03
Experiments use Qwen3-4B-Instruct and Qwen3-30B-A3B-Instruct. The held-out in-space evaluation averages over 18 fixed configurations excluded from the main training set, across ClawGym-Bench, JobBench-Easy and PinchBench. Pass@1 averages three independent evaluation runs, with GPT-5.4 used where rubric judging is required.
- 04
For 4B GRPO, Table 1 reports in-space ClawGym-Bench of 52.78% versus 37.33% under single-harness training, and Claude Code performance of 45.37% versus 38.96%. These are absolute gains of 15.45 and 6.41 percentage points. Its six-cell average across three benchmarks and two harness settings rises from 29.81% to 39.04%.
- 05
The corresponding six-cell averages improve from 33.91% to 36.93% for 4B PPO, from 37.29% to 42.78% for 30B-A3B GRPO, and from 36.31% to 41.89% for 30B-A3B PPO. These are benchmark/harness averages, not one population-wide success rate.
- 06
On unseen Claude Code, GRPO Harness Scaling averages 37.22% for 4B and 41.20% for 30B-A3B, versus 33.38% and 36.28% for GRPO trained through OpenClaw. The report specifies OpenClaw v2026.4.5 for black-box training and Claude Code v2.1.205 for transfer evaluation; the evaluated policy remains the Qwen model.
- 07
The structured 18-harness set beats a random 18-harness set and the full 243-configuration setting at the tested fixed budget. Two harnesses per task outperform both one and four in the reported ablations. A Repeat-2 control reuses the same harness on the same two-step schedule and performs worse than varying the harness, supporting a benefit beyond simple task repetition.
Evidence & limits
The full 243-harness ablation includes the white-box evaluation configurations in training. Its result is a fixed-budget stress test, not held-out generalization, and should not be compared as if all settings have the same train/test separation.
Matched update and rollout counts do not establish matched GPU-hours, generated tokens, latency or dollar cost. Context compaction changes segment counts and execution configurations can change rollout lengths; no general compute-efficiency gain is established.
Three evaluation runs do not measure robustness across independent RL training seeds. The main tables give point estimates without training-seed confidence intervals, so small differences should not be treated as statistically established.
External harness transfer is tested with specific software versions and task suites. Claude Code here is an execution harness for the trained Qwen policy, not a claim about improving or outperforming an underlying Claude model.
Only 18 held-out combinations are evaluated in the main in-space study, rather than every unseen configuration. Pairwise-balanced coverage does not guarantee robustness to arbitrary higher-order interactions or new component types.
The two backbones include a 30B-A3B MoE model, and their context limits and GPU allocations differ. The experiments do not establish a dense-model size scaling law or monotonically improving returns from harness count.
Black-box SFT reuses 12,800 fixed trajectories for three epochs, while RL collects online trajectories for 300 updates. Approximately equal trajectory exposures do not equate their data distributions, learning objectives or computational cost.
Scaling dimensions
Models, methods & benchmarks
- Models
- Qwen3-4B-Instruct, Qwen3-30B-A3B-Instruct
- Algorithms
- Harness Scaling, GRPO, PPO
- Benchmarks
- ClawGym-Bench, JobBench-Easy, PinchBench
Topics