TL;DR
KnowGPO groups search-agent steps by accumulated task-relevant knowledge and rewards newly acquired facts. Controlled Qwen3-8B comparisons show gains over GRPO on four evaluation splits, but the method requires reference answers, successful training trajectories, and an external LLM verifier. Several narrative claims in the first version disagree with its tables.
Why it matters
Search trajectories can collect the same useful facts through different pages and queries. A knowledge-state representation offers a reusable credit-assignment interface when raw observations rarely repeat, while making the quality and cost of the verifier central to the training recipe.
Key findings
- 01
Knowledge conditions are constructed in two stages: decompose a question into atomic constraints with entity placeholders, then ground them using its reference answer and a successful rejection-sampled trajectory. This is privileged training supervision rather than an answer-free reward source.
- 02
A DeepSeek-v3.2 verifier tracks satisfied conditions, accumulated by logical OR. Appendix A groups non-terminal steps by their preceding knowledge state and normalizes incremental fact counts within each group. Step and episode advantages are mixed with weight 0.3 for the step term and broadcast over action and reasoning tokens in a clipped policy objective.
- 03
The shared Qwen3-8B initialization is fine-tuned on about 8,100 search trajectories. RL uses 649 selected training queries, 32 prompts per batch and 16 rollouts per prompt for three epochs on 64 H20 GPUs. The search environment routes Chinese queries to Xhub and English queries to Google, with a Qwen3-8B snippet summarizer; context is extended to 128K via YaRN.
- 04
Tables 1–2 report KnowGPO accuracy of 38.8% on BrowseComp-ZH, 40.2% on text-only GAIA, 48.0% on Xbench-2505 and 36.0% on Xbench-2510. Relative to the matched GRPO rows, these are absolute gains of 5.9, 5.2, 2.0 and 2.0 percentage points. GPT-4o judges answer matches; the paper reports means over three evaluation runs.
- 05
Against the strongest GiGPO or IGPO row on each split, the corresponding gains are 1.5, 2.3, 2.0 and 1.0 percentage points. These controlled comparisons share SFT initialization, data and rollout settings; external agent leaderboard rows do not share those controls.
- 06
On 5,922 valid trajectories with 55,571 steps from 500 queries, knowledge grouping places 97.8% of steps in non-singleton groups, versus 51.8% for observation grouping. This is a grouping-coverage diagnostic, not an accuracy or compute-efficiency measurement.
- 07
The timing breakdown lists 356 seconds of verification per iteration, versus 2,574 seconds for generation, 583 for updates and 310 for probability computation. The authors overlap verification with the next rollout; 2.76 seconds of gain/group bookkeeping is small, but verifier resources and offline condition construction remain real costs.
Evidence & limits
The first-version prose is internally inconsistent: it claims that 48.0% beats a MiroThinker score of 60.6%, while Table 2 shows the opposite. Its headline 3.8–4.2-point claim is not a uniform per-split margin over the strongest baseline. This note uses table values and does not repeat those claims.
The grouping equation in the main text uses the current state, whereas Algorithm 1 and the explanatory text use the preceding state. This note follows the algorithm; exact reproduction should resolve that indexing discrepancy.
The zero-step-weight ablation differs from GRPO despite the stated objective reducing to an outcome-only update. The paper does not resolve this discrepancy, so the suggested implicit regularization explanation is not established.
External LLM verification is not a deterministic correctness oracle. The reported 94.7% per-condition agreement comes from 200 annotated query-trajectory pairs. Monotonic OR can preserve false positives, and the verifier prompt allows common-knowledge exceptions to retrieval evidence.
Three evaluation runs do not establish robustness across independent training seeds. No uncertainty intervals accompany the main tables, and dynamic search results, summarization and LLM judging can affect reproducibility.
All controlled experiments use one 8B backbone. Larger external agents differ in data, tools and context; these results establish neither a model-size scaling law nor broad superiority to 32B systems.
Asynchronous overlap can reduce elapsed time without eliminating computation or API expenditure. The paper does not establish equal total compute or end-to-end dollar cost against GRPO, including successful reference collection and condition construction.
Scaling dimensions
Models, methods & benchmarks
- Models
- Qwen3-8B, DeepSeek-v3.2
- Algorithms
- KnowGPO, GRPO, GiGPO, IGPO
- Benchmarks
- BrowseComp-ZH, GAIA, Xbench-2505, Xbench-2510
Topics