Sourced guide

Prompt caching: cut your LLM input cost up to 90%

One line of configuration turns a large, repeated context from a full-price bill into a tenth of the cost. Here is the exact mechanism, the numbers, and how to switch it on.

The problem

Every call to a large language model re-reads its whole input. If you send a big system prompt, a set of tool definitions, or a reference document on every request, you pay full input price for that same block again and again. For a long-running assistant or a chatbot with long history, the re-read — not the new message — is where the money goes.

The mechanism

Prompt caching stores a fixed prefix of your request and reuses it across calls. That prefix is usually a large system prompt or a set of tool definitions. The provider bills the block once at full price to write it, then serves it from cache on the requests that follow.

The economics are precise. A cache read costs a tenth of the normal input-token price, so you save ninety percent on the cached portion. Writing the cache costs 1.25 times the base input price for a five-minute cache. A large, stable prefix reused a few times inside that window pays for itself almost at once.

Turn it on

The simplest form adds one parameter — cache_control set to ephemeral — and the API caches the last eligible block automatically. For finer control you place cache_control on a specific system or text block.

The default cache lives five minutes and refreshes each time it is used inside that window at no extra cost. A one-hour cache is available at twice the base input price when you need a longer horizon.

Caching only activates above a per-model minimum — on the order of a thousand tokens for the mid-tier models, fewer for the smallest. Below that, the request runs uncached with no error, so check the response.

Confirm it worked

The usage object on the response tells the truth: cache_read_input_tokens are billed at the 10% rate, cache_creation_input_tokens carry the write premium. If both read as zero, nothing was cached — usually because the prefix was under the minimum or changed between calls.

Where it pays most

Any workload that resends the same large context benefits. An assistant that carries heavy system prompts and tools is the clearest case; a support bot with a long running history is another. The more often the prefix repeats inside the window, the more that single line of configuration saves you.