What Is Prompt Caching? (Cheaper Repeat Requests)

Updated July 23, 2026 · Data as of 2026-07-09

Prompt caching is a provider feature that stores the processed form of a repeated section of your request, usually a long instruction block or an attached document, so that later requests reusing that section are cheaper and faster.

The saving comes from where the repetition sits. Providers can reuse work only on an identical prefix, meaning the text at the very start of the request, byte for byte. Change one character near the top and the cache no longer applies to anything after it. This is why the practical advice is always the same: put the stable material first, your long instructions, the reference document, the code file, and put the part that changes, your actual question, at the end.

The discount is substantial where it applies. Cached input is commonly billed at a small fraction of the normal input rate, and it also returns faster because the expensive step was already done. Entries expire after a short idle period, typically minutes, so the benefit lands on bursts of related requests rather than on occasional use spread through a day.

This is mostly invisible in consumer products. If you use a chat interface or a coding assistant, the vendor may be using caching internally, and you see it as lower prices or quicker replies rather than as a setting. It becomes something you manage directly when you build against an API, where structuring the request well is a real and repeatable saving.

Why it matters when you choose a tool

For most buyers this is background detail rather than a criterion, and it should not decide a subscription. It becomes relevant if you are building something that sends the same long context repeatedly, a support assistant with a fixed knowledge base, a code reviewer that reloads the same standards document, a batch job over many similar inputs. In those cases caching can change the economics enough to alter which provider is cheapest, so it belongs in the comparison rather than in the footnotes.

Frequently asked questions

Do I need to turn prompt caching on?

In consumer products, no, and usually you cannot see it at all. On developer APIs it varies: some providers apply it automatically to qualifying requests, others require you to mark which part of the request is cacheable.

How long does a cache entry last?

Typically minutes of idle time rather than hours, with some providers offering longer retention at a price. It is designed for bursts of related requests, not for once-a-day reuse.

Does caching change the answer I get?

No. It reuses computation over identical input. The model's response is generated fresh each time, so two identical requests can still return different wording.

Back to the glossary · Take the quiz · How we score tools