When Does Prompt Caching Actually Kick In? Thresholds, the Price Change, and a Quick Demo

In our last post on prompt order, we covered the one rule that decides whether your input gets cached: put the static parts first, the changing parts last, and keep the prefix byte-for-byte stable.

But there's a catch nobody mentions. You can order your prompt perfectly and still cache nothing — because caching doesn't switch on until your prompt crosses a minimum size. Below that line, the discount simply doesn't exist. Above it, a big chunk of your prompt suddenly costs a fraction of the price.

It behaves like a cliff, not a slope. This post is about exactly where that cliff is, what happens to the bill when you step over it, and a simple way to see it.

1. The threshold: caching has a minimum prefix length

Every provider sets a floor. If your stable prefix is shorter than that floor, the provider won't bother caching it — there's nothing to gain from reusing a tiny prefix, so the request is processed at full price even if it's identical to the last one.

Order makes your prefix cacheable. Size makes it cached. You need both — a perfectly ordered 800-token prompt still caches nothing.

Here's where each provider draws the line:

Two things worth pulling out of that table:

• OpenAI matches in 128-token increments. A 1,100-token prefix caches the first 1,024 of it (it rounds down to the nearest step), so the last few tokens before a step boundary are "wasted" from a caching standpoint.
• Anthropic's floor moves with the model. The same 3,000-token prompt caches fine on a model with a 1,024 or 2,048 minimum, but silently caches nothing on one with a 4,096 minimum — no error, just a cached-token count that stays at zero. Always check your specific model's minimum.

And the most important consequence: when your prefix is just under the line, adding a little more static content can make it cheaper, not more expensive — because crossing the threshold unlocks the discount on the whole prefix.

2. The price change: what happens at the line

Below the threshold, every token is billed at the normal input rate. The instant you cross it, the part of your prompt that repeats splits into two new prices:

The read discount is roughly the same everywhere — cached input costs about a tenth of fresh input. Where providers differ is the cost of that first write and how long the cache survives:

That write premium on Claude is the one number people forget. Because the first call costs a little more than usual, caching is a bet that pays off only with repetition:

• On the 5-minute window, the cache pays for itself after about 2 reads (1.25× to write + 0.1× to read beats paying 1× twice).
• On the 1-hour window, the write costs 2×, so you need about 3 reads to come out ahead — but the entry survives much longer gaps in traffic.

So caching is a win for repeated calls against the same prefix, and a small loss for a true one-off. On a busy endpoint sending the same system prompt all day, it pays for itself almost instantly.

3. A quick demo: watch the cliff

Picture a single dial: the size of your stable prefix. Drag it from small to large and the price doesn't ease down — it falls off a ledge the moment you cross the minimum.

Now make it concrete. Say your shared system prompt is 1,000 tokens, ordered perfectly — static instructions first, the user's message last:

The logic and order are identical; the only difference is that one crossed the caching threshold and the other didn't, which is the difference between paying full price on every call and paying a tenth.

The takeaway

Prompt order gets you eligible for caching. Thresholds decide whether it actually happens. To make sure you land on the cheap side of the cliff:

• Clear the minimum with room to spare. Don't park your shared prefix at ~1,000 tokens hoping to cache — push it comfortably past the floor.
• Know your model's exact minimum. It ranges from 1,024 to 4,096 tokens, and on Claude it changes by model. The same prompt can cache on one model and silently miss on another.
• Budget for the cold write. On Claude the first call costs 1.25–2× — caching is for repeated calls, not one-offs. It breaks even after 2–3 reads.
• Mind the TTL. Entries expire (5 minutes by default). Bursty or widely-spaced traffic may keep missing a warm cache — use the longer 1-hour window, or pre-warm the cache, if your calls are spread out.
• Watch the cached-token counters. Every provider reports cached tokens in the usage response. If that number stays at zero across repeated calls, you're either below the threshold or something in the prefix is changing.

Two prompts can be functionally identical, ordered identically, and still cost wildly different amounts — because one cleared the threshold and the other didn't. Get both the order and the size right, and the cheapest part of your bill takes care of itself.

How we optimize LLM usage — and how you can too

Thresholds and prompt order are two levers on the same axis: paying less per call. Picking the right model for each request is the other axis — and that one we open-sourced. TryAii-RE is our free, open-source AI model routing engine: instead of hard-wiring every call to a single expensive flagship, it routes each request to the model that actually fits the task, balancing cost, latency, and quality. Pair smart routing with a cache-friendly prompt that clears the threshold, and you're optimizing on every axis at once.

It installs in one line for Python or Node, runs anywhere, and is free to use. Grab it below: