Kimi K3 puts explicit prompt caching on Bedrock behind a 1,024-token minimum prefix

Open weights at this size still leave you on someone else's cluster. Moonshot AI calls K3 the first open model to reach 2.8 trillion parameters ^[2]. At one byte per parameter, that is roughly 2.8 TB of weights before you allocate anything for a million-token KV cache ^[15]. Few teams keep that much accelerator memory idle as a fallback.

Caching here is a billing decision you express in the prompt, and AWS calls K3 the first open-weight model on Bedrock to support it explicitly ^[4]. You mark the end of a reusable prefix with a `prompt_cache_breakpoint` on a supported input content block, and the prefix has to run at least 1,024 tokens ^[5]. In explicit mode the tokens written to cache are billed at a higher rate and held for at least 30 minutes ^[6]. Matching later requests hit the cache and are billed less on input ^[7]. Call the write multiplier w and the hit rate r, both relative to the normal input price: the prefix has to be reused more than (w - r) / (1 - r) times inside that 30-minute window before caching beats resending it ^[17]. An agent grinding through one repository clears that easily. A per-tenant chat surface whose system prompt changes every request never will.

The platform work underneath is the part I would call good engineering. AWS says the 2026 additions, tool calling, structured output, reasoning, response streaming and the Responses and Chat Completions APIs, are platform capabilities and not per-model integrations ^[11]. The `bedrock-runtime` endpoint answers the OpenAI-compatible Responses and Chat Completions APIs as well as Invoke and Converse ^[12], so an existing agent changes a model ID. The prerequisites are less portable: the post lists `bedrock:InvokeModel`, `bedrock:InvokeModelWithResponseStream` and `bedrock:CreateInference` ^[10]. An invoke-only role needs the third before the first call.

On the 2.5x, AWS's post credits Moonshot AI for the capability claims and reports "an approximate 2.5x improvement in scaling efficiency" over Kimi K2, alongside native vision and the 1-million-token window ^[3]. The post leaves scaling efficiency undefined and the workload it was measured on unnamed ^[18]. For that figure to reach your invoice it would have to be a statement about tokens per dollar at your context length and your concurrency.

AWS recommends `global.moonshotai.kimi-k3` for workloads without regional restrictions, routing each request to any supported commercial Region ^[8], and says global cross-Region inference costs approximately 10% less than a geographic profile ^[9]. Inverted, `us.moonshotai.kimi-k3` costs about 11% more per request for the same model ^[16].

The security envelope is the other half of what you are buying. AWS says data is processed within the AWS data boundary, is not shared with the model provider and is not used to train the model, with zero data retention always enabled for inference requests and zero operator access blocking even AWS operators from prompts and completions during inference ^[13].