Kimi K3: Open Frontier Intelligence

TL;DR

A technical report on Kimi K3, a 2.8T-parameter sparse multimodal model with 104B active parameters and a one-million-token context window, post-trained with reinforcement learning across general, agentic, coding, and reasoning domains.

Why it matters

Kimi K3 connects model sparsity, long-context architecture, policy training, rollout infrastructure, and persistent execution environments in one scaling recipe. It is especially relevant for showing what agent RL requires when trajectories and sandbox state extend toward a million tokens.

Key findings

1. 01

   The authors combine Kimi Delta Attention, Attention Residuals, and Stable LatentMoE, reporting roughly 2.5× better overall scaling efficiency than Kimi K2.

2. 02

   Reinforcement learning spans long-horizon coding, general agents, reasoning, and knowledge tasks at multiple reasoning-effort levels before the capabilities are consolidated into one model.

3. 03

   The million-token agentic RL system uses partial rollouts, external KV-cache retention, adaptive throttling, and resumable microVM sandboxes to preserve long-lived model and environment state.

Scaling dimensions

Models, methods & benchmarks

Models

Kimi K3, Kimi K2

Algorithms

Kimi Delta Attention, Attention Residuals, Stable LatentMoE, Agentic reinforcement learning

Topics