TL;DR
Xiaomi’s CodeMidas turns implemented functionality in existing codebases into executable coding RL tasks. Training MiMo-V2.5 on 5,545 filtered environments improves five external benchmarks, while a 3,000-task filtered subset beats an 8,000-task unfiltered baseline on three evaluations.
Why it matters
Scaling coding RL requires a growing supply of reliable environments, not just more rollouts. CodeMidas uses source code as its only task-specific input, extending task construction beyond recorded issues and commits. Its ablation makes environment quality a concrete part of the scaling discussion.
Key findings
- 01
The pipeline derives behavioral specifications from existing functionality, removes target implementations, constructs tests grounded in execution of the original code, and checks environment consistency. Post-rollout filtering probes solution leakage, reviews verifier disagreements, and retains tasks with both successful and failed screening attempts.
- 02
The resulting training set contains 5,545 tasks from 3,185 codebases, spanning 23 programming languages and 15 technical domains. MiMo-V2.5 is trained with GRPO, binary execution rewards, batch size 32, and 32 rollouts per task; hidden verifiers are injected only at grading.
- 03
DeepSWE pass rate rises from 10.0% to 21.7% (+11.7 percentage points), and Terminal-Bench v2.1 rises from 63.7% to 72.2% (+8.5 points). ProgramBench Almost Solved rises from 4.5 to 21.5 (+17.0 points); this metric requires at least 95% of tests to pass, rather than complete task success.
- 04
With identical training configurations, increasing filtered task pools from 1,000 to 3,000 to 5,545 raises DeepSWE scores from 17.57 to 19.05 to 21.70. Even the 3,000-task subset outperforms the vanilla 8,000-task baseline on SWE-bench Pro, DeepSWE, and CodeMidas Val.
Evidence & limits
The training study uses MiMo-V2.5 and three filtered dataset sizes. It supports a task-scaling trend in this setting, not a universal scaling law across model sizes or training budgets.
The 8,000-task baseline omits environment cleaning, execution consistency checks, and all three post-rollout filters. The comparison measures their combined effect and does not isolate the contribution of each filter.
Environment construction itself consumes agentic compute. These results do not establish that the complete construction-and-training pipeline is cheaper at matched performance.
Screening difficulty depends on the model and attempt budget, and generated tests can still miss defects. The reported association between agent-written checks and higher success is observational, not proof that self-verification causes the gain.
Scaling dimensions
Models, methods & benchmarks
- Models
- MiMo-V2.5
- Algorithms
- GRPO, Execution-based rewards, Environment filtering
- Benchmarks
- SWE-bench Pro, DeepSWE, ProgramBench, RepoZero C2Rust, Terminal-Bench v2.1, CodeMidas Val
Topics