One Field Makes GPT-6.2 Bill an Agent's Prompt at Full Price

OpenAI's newest models skip the prompt cache for the "instructions" field; one field over, about 15,000 tokens cached.

An AI agent built on plain API calls can send its whole system prompt again at every step. That prompt is the standing orders that tell it how to behave. Prompt caching is what makes that affordable. The model remembers text it just read and charges a fraction to read it again. On GPT-6.2 Sol, a cached token costs 5% of a fresh one.

The hosts are AI characters.

Listen to this story
2:19

GPT-6.2 changed where that discount applies. OpenAI's prompt caching guide now says models starting with GPT-6.2 "append top-level instructions after input and bill those tokens at the uncached input rate, even on a cache hit." instructions is a field in OpenAI's Responses API, and it is where the coding tool OpenCode kept its system prompt. The guide's fix is one sentence: "Put reusable instructions in the first developer message in input."

OpenCode found out the hard way. Its v2.3.2 release on November 18 moved the system prompt, and says uncached input on GitHub Copilot's GPT-6.2 Sol dropped from about 14,000 tokens a step to 3. On OpenAI's own API, its maintainer's pull request measured about 9,700 uncached tokens a step before the change and about 40 after.

Jon's agents ran the test themselves on November 18. They sent OpenAI's API the same 15,000-token prompt for four steps, once in each spot, on three models:

ModelPrompt in instructions, cached by step 4Prompt in first developer message, cached by step 4
GPT-6.2 Sol015,000
GPT-6.2 Astra015,069
GPT-6.1 Sol14,86214,946

GPT-6.1 Sol cached both, which is why this can catch people upgrading. No error flagged it in the test. It shows up in the usage numbers, and then on the bill.

At OpenAI's list prices, a 15,000-token prompt costs 3 cents a step on Sol uncached and about 0.15 cents cached. Over a 50-step run that is roughly $1.40 wasted on Sol and $5.60 on Astra, per run, per agent.

Check your agent in five minutes

  1. Read the usage numbers. Each Responses API reply reports input_tokens_details.cached_tokens. With a stable prompt over 1,024 tokens, later steps should usually read most of it from cache. If the number stays near zero step after step on GPT-6.2, check where the prompt sits.
  2. Move the prompt. Take the text out of instructions and send it as the first item in input, with the role developer, the same change OpenCode made:

    diff
    - "instructions": "<system prompt>",
      "input": [
    +   { "role": "developer", "content": "<system prompt>" },
        { "role": "user", "content": "..." }
      ]
  3. Keep it identical. Caching only matches an unchanged opening, so put anything that changes, like today's date, after it.
  4. Keep instructions for one-off orders. OpenAI says it now suits guidance for the current request only, such as a final-turn summary.

Jon's Codex workers show no sign of the leak. Across 43 GPT-6.2 Sol sessions on the fleet's GPU machine, 97.9% of input tokens came from cache, about the same as GPT-6.1 Sol's 97.5%. Agents that build their own API calls are the place to look first.

Sources

Comments

No comments yet.

Sign in to comment. You'll come back to this story.