kimi k3 model

Kimi K3 Explained: Inside the World’s First Open 3T-Class AI Model

by

If you have been following the AI model race in 2026, one name keeps showing up in benchmark tables next to GPT and Claude: Kimi. Kimi is not a single model. It is a family of large language models plus a consumer chatbot built by Moonshot AI, a Beijing based startup founded in March 2023 and backed by investors including Alibaba and IDG Capital.

On 16 July 2026, Moonshot released Kimi K3, a 2.8 trillion parameter model with a 1 million token context window and native vision. Ten days later, on the evening of 26 July, the company published the full weights on Hugging Face. That single act made K3 the largest open weight model ever released to the public.

This article walks through the whole story step by step: the lineage, the architecture, the benchmark numbers with their caveats, the pricing math, working API code, and what it actually takes to self host a 2.8 trillion parameter model. By the end you should be able to decide whether K3 belongs in your stack, and explain to a colleague exactly why it is architecturally interesting rather than just large.

Step 1: The Kimi timeline

Kimi did not appear overnight. Moonshot has shipped continuously since late 2023, and each release moved one specific dial. Understanding the sequence makes K3 much easier to place.

DateReleaseWhat changed
Oct 2023Kimi chatbot (closed beta)First product launch
Nov 2023Kimi public release128K token context, unusually large for the time
Mar 2024Long context betaRoughly 2 million character context window
Jul 2024Context cachingPublic beta of prompt prefix caching
Oct 2024Kimi Explore EditionAutonomous search, monthly active users passed 36 million
Apr 2025Kimi-VLSmall open vision language model, ~2.8B active params
Jul 2025Kimi K21T parameter MoE, 32B active, 384 experts, 256K context
Nov 2025Kimi K2 ThinkingOpen reasoning variant
Oct 2025Kimi LinearLinear attention research line that fed into KDA
Jan 2026Kimi K2.5Unified text, image and video multimodality
Apr 2026Kimi K2.6Coding focused refresh
Jul 2026Kimi K32.8T params, 1M context, native multimodal, open weights

Two things stand out. First, the jump from K2 to K3 is not a straight scale up: the architecture was largely redesigned. Second, Moonshot deprecated aggressively. The kimi-k2 API series was retired on 25 May 2026, and kimi-latest was retired on 28 January 2026, so anyone building against those names has already had to migrate.

Step 2: The headline specifications

AttributeKimi K2Kimi K3
Total parameters1 trillion2.8 trillion
Active parameters per token32 billion104 billion
Routed experts384 plus 1 shared896, with 16 active per token
AttentionMulti-head Latent Attention (MLA)Kimi Delta Attention plus interleaved Gated MLA
Context window256K tokens1,048,576 tokens
ModalityTextText, image, video, natively trained together
Numeric formatINT4MXFP4 weights, MXFP8 activations
OptimizerMuon with weight clippingPer-Head Muon with weight clipping
LicenseModified MITKimi K3 License (MIT derived)

The sparsity ratio is the number worth internalising. K3 holds 2.8 trillion parameters in memory but only touches about 104 billion of them for any given token. That is roughly 3.7 percent activation. You get the knowledge capacity of a very large model at the per token compute cost of a much smaller one, which is exactly the trade that makes trillion scale models economically viable at all.

Step 3: The architecture, piece by piece

Moonshot’s own framing is that K3 delivers about 2.5 times better scaling efficiency than K2, meaning it reaches comparable quality using significantly fewer FLOPs. That gain comes from six named changes, not one.

flowchart TD
    A[Input tokens: text, image, video] --> B[Shared multimodal embedding]
    B --> C[Block 1: KDA layer]
    C --> D[Block 2: KDA layer]
    D --> E[Block 3: KDA layer]
    E --> F[Block 4: Gated MLA layer]
    F --> G[Stable LatentMoE router]
    G --> H[16 of 896 experts activated]
    H --> I[Shared expert]
    I --> J[AttnRes: selective retrieval from all earlier layers]
    J --> K[Repeat to depth N]
    K --> L[Output tokens]

3.1 Kimi Delta Attention (KDA)

KDA is a hybrid linear attention mechanism, a delta rule recurrence with a per channel forget gate. Queries and keys pass through a short convolution, a Swish activation, and L2 normalisation. Each KDA layer then maintains a recurrent state of shape d_k by d_v that is updated per chunk: parallel within a chunk, recurrent across chunks.

The reason this matters is memory, not elegance. In a standard transformer the KV cache grows linearly with context length. At 1 million tokens the cache dominates VRAM and becomes the binding constraint on how many concurrent requests a GPU can serve. KDA layers carry a constant size recurrent state instead. Only the periodically interleaved Gated MLA layers hold a global KV cache, so the total cache footprint at 1M tokens is a fraction of what a pure attention stack would need.

Here is a simple way to see the difference:

def kv_cache_gb(tokens, layers, kv_heads, head_dim, bytes_per_val=2):
    """Approximate KV cache size in GB for a standard transformer."""
    return (2 * tokens * layers * kv_heads * head_dim * bytes_per_val) / (1024 ** 3)

# Pure attention stack, all layers cached
full = kv_cache_gb(tokens=1_000_000, layers=64, kv_heads=8, head_dim=128)

# Hybrid stack, only every 4th layer holds a global cache
hybrid = kv_cache_gb(tokens=1_000_000, layers=16, kv_heads=8, head_dim=128)

print(f"full attention : {full:8.1f} GB")
print(f"hybrid (1 in 4): {hybrid:8.1f} GB")
print(f"reduction      : {(1 - hybrid / full) * 100:6.1f} %")

full attention :    244.1 GB
hybrid (1 in 4):     61.0 GB
reduction      :   75.0 %

Those layer counts are illustrative rather than K3’s exact configuration, but the shape of the saving is real, and it is the reason Moonshot can price a 1M context model competitively. Because KDA breaks the assumptions of conventional prefix caching, Moonshot contributed a KDA aware prefill cache implementation upstream to vLLM and released it alongside the model.

3.2 Attention Residuals (AttnRes)

Standard residual connections accumulate layer outputs uniformly: every layer adds its contribution to a single running stream. AttnRes replaces that with selective retrieval. Each layer can attend to representations from any preceding layer rather than only the immediately previous one.

In a mixture of experts model this is especially useful, because different experts fire at different depths. A layer 40 expert that needs a signal computed at layer 6 can now reach for it directly instead of hoping it survived 34 rounds of accumulation.

3.3 Stable LatentMoE and Quantile Balancing

K3 more than doubles the expert count to 896 while activating only 16. At that sparsity, routing stops being a detail and becomes a primary failure mode: a handful of popular experts get hammered while most sit idle, throughput collapses, and training destabilises.

Quantile Balancing (QB) solves this by deriving expert allocation directly from the quantiles of the router scores. There is no auxiliary balancing loss and no sensitive balancing hyperparameter to tune. Token distribution across experts falls out of the score distribution itself.

Moonshot pairs this with a fully balanced expert parallel training method that uses static tensor shapes and keeps host synchronisation off the critical path, so one overloaded expert cannot stall an entire training step across thousands of accelerators.

3.4 Per-Head Muon, SiTU-GLU, and Gated MLA

Three smaller changes round out the stack:

  • Per-Head Muon extends the Muon optimizer so each attention head is optimised independently, giving more adaptive learning rates at scale. K2’s weight clipping mechanism is retained.
  • SiTU-GLU (Sigmoid Tanh Unit) is a custom activation replacing the standard GeLU or SwiGLU path. Its job is preventing activation explosions, with RMSNorm stabilising routed experts.
  • Gated MLA adds gating to Multi-head Latent Attention, improving attention selectivity in the layers that still hold a global cache.

3.5 Native multimodality

Most vision language models bolt a pretrained encoder such as CLIP or SigLIP onto a finished language model, then run an alignment stage. K3 does not. It uses MoonViT-V2, a vision encoder trained from scratch jointly with the language model. Visual and textual tokens are interleaved inside a single next token prediction objective from the very first training step.

Moonshot reports that this jointly trained approach improved training stability while matching conventional approaches on vision benchmarks. Practically, it is why K3 can do “vision in the loop” coding: write frontend code, take a screenshot, look at it, and iterate, all inside one model with no handoff.

3.6 Quantization aware training in MXFP4

K3 applies quantization aware training from the supervised fine tuning stage onward, using MXFP4 weights (4 bit floating point with per block scaling) and MXFP8 activations. This is a meaningful distinction from post training quantization: the model learns to compensate for quantization error while training, so quality degradation is far smaller than a naive 4 bit conversion. MXFP4 is natively supported on NVIDIA Blackwell and AMD MI400 hardware, so the format is a deployment advantage rather than a compatibility tax.

Step 4: Benchmarks, and how to read them honestly

Moonshot’s own positioning is refreshingly unarrogant. Their blog states plainly that overall performance still trails Claude Fable 5 and GPT-5.6 Sol, while claiming frontier level results across their evaluation suite.

BenchmarkKimi K3Claude Fable 5GPT-5.6 SolClaude Opus 4.8
SWE-bench Verified76.8–––
DeepSWE67.570.073.059.0
FrontierSWE81.286.671.366.7
Terminal-Bench 2.188.3–88.8–
Program Bench77.876.877.6–
SWE Marathon42.035.039.0–
BrowseComp91.288.0–84.x
HLE-Full43.553.3––
GPQA Diamond93.5–––
MMMU-Pro81.6–––

Three independent signals are worth more than the vendor table:

  1. LMArena Frontend Code Arena: K3 took the number one spot with an Elo of 1,679, jumping 17 places from K2.6’s rank of 18, and beating Claude Fable 5 in roughly 76 percent of blind head to head developer matchups. It placed first in six of seven frontend domains. Blind human voting is harder to game than a static test set.
  2. AA-Briefcase (Artificial Analysis, agentic knowledge work): K3 scored 1,548 Elo, second only to Fable 5 at 1,583, and ahead of GPT-5.6 Sol at 1,495. That is a gain of over 700 Elo against K2.6.
  3. Independent verification gaps: on Terminal-Bench 2.1, the vendor reported 88.3 using Moonshot’s own Kimi Code harness compares against roughly 85 measured independently by Artificial Analysis on their harness.

That last point is the critical caveat. A benchmark score is never just a model. It is a model plus a harness plus specific settings. Moonshot runs most coding benchmarks on its own Kimi Code harness while competitors run on Claude Code or Codex, and the footnotes say so explicitly, including that Fable 5 hit fallbacks on 35 percent of tasks in their SWE Marathon evaluation.

The honest summary: K3 is strongest on frontend code, sustained agentic execution, and long context browsing. It is weakest, relative to the top proprietary models, on deep general reasoning such as Humanity’s Last Exam, and on open ended conversation.

Step 5: Pricing and cost math

Token typePrice per million tokens
Input, cache hit$0.30
Input, cache miss$3.00
Output$15.00

There is no tiering by context length, which is unusual for a 1M context model. Moonshot’s inference stack (Mooncake disaggregated architecture) reportedly achieves a cache hit rate above 90 percent on coding workloads, and cache hits cost a tenth of misses, so the effective input price in a repeat prompt workload is far below the headline $3.

def k3_cost(cached_in, uncached_in, out, hit_rate=0.9):
    """Estimate Kimi K3 API cost in USD for a workload, token counts in millions."""
    return round(cached_in * 0.30 + uncached_in * 3.00 + out * 15.00, 2)

# A coding agent: 50M input tokens, 90% served from cache, 4M output tokens
total_in = 50
cost = k3_cost(cached_in=total_in * 0.9, uncached_in=total_in * 0.1, out=4)
print(f"estimated cost: ${cost}")   # estimated cost: $88.5

Note the counterweight flagged by independent testers: Artificial Analysis measured high output token consumption and slower than median generation speed for K3. Since K3 always thinks and reasoning tokens are billed as output at $15 per million, a model that reasons verbosely can cost more in practice than its per token rate suggests. Measure your own workload before committing.

Step 6: Using the API

The Kimi API is OpenAI compatible, so the standard SDK works with a swapped base URL. The flagship model unlocks after a minimum top up of $1.

python3 -m pip install --upgrade 'openai>=1.0'

import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["MOONSHOT_API_KEY"],
    base_url="https://api.moonshot.ai/v1",
)

completion = client.chat.completions.create(
    model="kimi-k3",
    reasoning_effort="max",              # low | high | max, default max
    messages=[{"role": "user", "content": "Prove that the square root of 2 is irrational."}],
)

print(completion.choices[0].message.content)

Streaming separates the chain of thought from the final answer into two distinct delta fields:

stream = client.chat.completions.create(
    model="kimi-k3",
    messages=[{"role": "user", "content": "Explain why the sky is blue."}],
    stream=True,
)

for chunk in stream:
    delta = chunk.choices[0].delta
    reasoning = getattr(delta, "reasoning_content", None)
    if reasoning:
        print(reasoning, end="", flush=True)
    if delta.content:
        print(delta.content, end="", flush=True)

Vision input requires base64 or an uploaded file reference, and content must be an array of objects rather than a string. Public image URLs are not supported:

import base64
from pathlib import Path

image_data = base64.b64encode(Path("image.png").read_bytes()).decode()

completion = client.chat.completions.create(
    model="kimi-k3",
    messages=[{
        "role": "user",
        "content": [
            {"type": "image_url",
             "image_url": {"url": f"data:image/png;base64,{image_data}"}},
            {"type": "text", "text": "Describe this image."},
        ],
    }],
)

Gotchas that will bite you

  • temperature, top_p, n, presence_penalty and frequency_penalty are fixed server side. Omit them entirely.
  • max_completion_tokens defaults to 131,072 and can go up to 1,048,576.
  • Always return the complete assistant message in multi turn and tool calling flows. Keeping only content breaks the model.
  • Prefix caching only engages when the previous request exceeded 256 prompt tokens.
  • Web search through official tools is being reworked and is not recommended for production right now.

Step 7: Self hosting the weights

The weights live at moonshotai/Kimi-K3 on Hugging Face under the Kimi K3 License, an MIT derived permissive licence. You may download, modify, fine tune, distribute and deploy commercially at no cost. Two conditions apply only at scale: a model as a service business exceeding $20 million revenue over any consecutive 12 months must sign a separate agreement, and any product with over 100 million monthly active users or over $20 million monthly revenue must display “Kimi K3” in its interface. Internal use is exempt. Read the actual LICENSE file in the repo before shipping commercially, since third party coverage of the terms has been inconsistent.

huggingface-cli download moonshotai/Kimi-K3 \
  --local-dir /data/models/kimi-k3 \
  --resume-download

pip install -U vllm

vllm serve /data/models/kimi-k3 \
  --tensor-parallel-size 8 \
  --max-model-len 131072 \
  --trust-remote-code

Hardware reality check: at MXFP4 the resident weights are on the order of 1.4 TB, and Moonshot recommends deploying on supernode configurations of 64 or more accelerators because inference efficiency benefits from larger high bandwidth communication domains. Reported download sizes vary across mirrors, so check the repo file list rather than trusting a blog figure. Both vLLM and SGLang shipped day zero serving support. Practical minimums reported by the community start around 8x H100 80GB, with 8x B300 or 16x B200 being far more comfortable. Cloud spot pricing has been quoted in the range of $45 to $110 per hour depending on configuration.

This is not a laptop model. If you cannot clear the cluster bar, hosted access through the Moonshot API, OpenRouter, Together, Fireworks and similar providers is the realistic path.

Step 8: Known limitations

Moonshot documents three limitations openly, and all three have operational consequences:

  1. Sensitivity to thinking history. K3 was trained in preserved thinking history mode. If your agent harness fails to pass back all historical reasoning content, or if you switch an in progress session from another model to K3, output quality can become highly unstable. Use a verified harness and do not swap models mid session.
  2. Excessive proactiveness. Training emphasised long horizon difficult tasks, so when K3 meets a minor obstacle or ambiguous intent it tends to act rather than ask. If you need the agent to stay inside defined boundaries, write explicit behavioural constraints into your system prompt or AGENTS.md.
  3. User experience gap. Moonshot itself acknowledges a noticeable UX gap compared with Claude Fable 5 and GPT-5.6 Sol, separate from raw capability.

Step 9: Should you use it?

Your use caseVerdict
Frontend and web UI generationStrong fit, ranked first on blind developer voting
Long horizon coding agents over big reposStrong fit, this is what it was optimised for
Very long context research and browsingStrong fit, 1M context with competitive pricing
Documents, spreadsheets, slide generationGood fit, second only to Fable 5 on AA-Briefcase
Multimodal work mixing code and screenshotsStrong fit, native vision is a genuine differentiator
Deepest general reasoning and hard science QAWeaker, Fable 5 leads clearly on HLE
Open ended conversation and creative writingWait for broader independent coverage
Data sovereignty or air gapped deploymentUniquely strong, since the weights are downloadable

The bigger picture

The interesting thing about Kimi K3 is not the parameter count. It is the claim underneath it: that a 2.5 times gain in intelligence per unit of compute came from redesigning attention, residuals, routing, activation, optimizer and numeric format together, rather than from throwing more GPUs at the same blueprint. Moonshot also opened parts of the surrounding stack, including attention kernels, an MoE communication library, and infrastructure for running agent environments at scale.

For anyone building on LLMs, the practical takeaway is a pricing one. A frontier adjacent model with downloadable weights sets a ceiling on what anyone can charge for comparable capability. Whether or not you deploy K3, its existence changes your negotiating position.

Sources

Found this useful? Pass it on to someone who is still comparing models on parameter count alone.

Leave a Comment