Running GPT-6 in production is expensive. Every API call costs tokens, tokens cost money, and if your agents are doing serious work — processing documents, running multi-step workflows, answering customer questions at scale — those costs add up fast.
OpenAI just made it significantly cheaper. The company launched an improved prompt caching system for the GPT-6 family that delivers up to 90% off on cached input tokens. For teams running AI at any real volume, that’s a budget conversation worth having.
How Prompt Caching Actually Works
Every time your code calls the GPT-6 API, you send a prompt. Most production prompts aren’t entirely unique — they start with the same system instructions, tool definitions, or reference documents, and then add the specific user input or task at the end. That shared opening, called a prefix, is exactly what caching targets.
OpenAI stores the processed version of that prefix on their servers. When your next request arrives with the same opening, the model skips re-processing it and serves the cached computation instead. Same quality output. Fraction of the cost.
The window is 30 minutes. Shared prefixes stay cached and eligible for reuse for 30 minutes after the last use. Structure your prompts so the stable content comes first and the variable content comes last, and you’ll hit those discounts consistently.
What’s New With GPT-6
Earlier versions of prompt caching required more manual setup and delivered less predictable results. GPT-6 changes that in a few concrete ways:
Higher default cache hit rates. The system is smarter about recognizing what can be reused without extra configuration from developers.
Prompt Caching Dashboard. A new interface in the OpenAI developer platform shows your cache hit rates over time and how changes to your application affect caching performance. Useful for tracking whether a code change accidentally broke your cache efficiency.
Diagnostics tool. When a cache miss happens unexpectedly, you can now investigate why — what changed in the request that prevented reuse, and how many tokens it cost you. This turns cache optimization from guesswork into an engineering discipline.
Cache prewarming. You can load your system instructions and tool definitions into the cache before the first user request arrives. Processing moves out of the user’s wait time and into your app’s startup sequence.
Reasoning adjustments without cache breaks. You can now tell the model to think harder or lighter on a specific task — useful in workflows where some steps need deep reasoning and others don’t — without invalidating the cached context shared across the session.
Who This Matters Most To
The 90% discount applies to the cached portion of your input tokens. Getting there requires thinking about prompt structure as an engineering decision:
- Are your system instructions at the top and consistent across requests?
- Are your tool definitions stable between calls, or do they change unnecessarily?
- Are you sending variable content (user input, task specifics) after the stable content, not before it?
Teams processing millions of tokens a day — customer support bots, document analysis pipelines, coding assistants — will see the biggest impact. But even smaller teams running AI workflows daily will notice a meaningful difference in their monthly bills.
The caching infrastructure is in place. Whether you capture the savings depends on how your prompts are engineered.
Want to explore how to build cost-efficient AI features for your business? Let’s talk.