TL;DR
MOPD-Router selects and weights teacher supervision at each token rather than assigning one domain teacher to a whole prompt. ExpertAlign uses each teacher’s shift from a shared pre-RL base to identify useful corrections, improving a nine-benchmark aggregate in four tested settings.
Why it matters
Domain labels are a coarse way to combine specialist models: a coding response may also need mathematical or instruction-following expertise. Token-level routing tests whether one student can absorb complementary signals without a separately trained router.
Key findings
- 01
The teacher pool contains three Qwen3-4B non-thinking specialists in math, code, and instruction following. Students are Qwen3-1.7B or Qwen3-4B; training uses either a 60K unlabeled prompt mixture or a 66K domain-labeled mixture, whose labels the router ignores.
- 02
ExpertAlign compares a teacher’s proposed correction to the student with its specialization relative to the shared base, on the student’s top-k token support. Positive alignment gates teacher selection, and cosine strength weights the retained signals.
- 03
For the 4B student on unlabeled data, ExpertAlign reaches 53.76 overall versus 47.88 for mean aggregation and 48.76 for Standard MOPD. The gains are 5.88 and 5.00 score points, respectively; overall is a macro-average of nine benchmark metrics, not one task-success rate.
- 04
On labeled data with the 4B student, the main result is 54.58 versus 50.63 for Standard MOPD. Across three training seeds in this setting, the mean advantage is 3.76 ± 0.32 points. ExpertAlign uses 716.9 versus 692.9 GPU-hours under the common step budget, while token-normalized cost is 1.18× the baseline.
Evidence & limits
The experiments use three same-family teachers sharing a pre-RL base. Larger or heterogeneous teacher pools and teachers trained from different bases are not validated here.
ExpertAlign needs an additional shared-base forward pass and routing statistics. Its measured 3.5% GPU-hour increase differs from its 18% token-normalized overhead because response lengths differ; neither establishes a compute-matched performance advantage.
Timing excludes initialization, model downloads, validation, and checkpoint I/O outside the per-step timer. Reported GPU-hours should not be treated as the complete operational cost.
Three-seed robustness is tested only in the same-size, domain-labeled setting. Evaluation bootstrap intervals in the other settings do not measure training-seed variability.
Standard MOPD on the originally unlabeled mixture receives LLM-generated domain labels. Math Avg@8, code Pass@1, and instruction-following prompt accuracy are aggregated with equal benchmark weights, so interpretation depends on this evaluation mix.
Scaling dimensions
Models, methods & benchmarks
- Models
- Qwen3-1.7B, Qwen3-4B
- Algorithms
- Multi-teacher on-policy distillation, ExpertAlign, Token-level routing
- Benchmarks
- AIME 2024, AIME 2025, HMMT 2025, HumanEval+, MBPP+, LiveCodeBench v6, IFEval, IFBench
Topics