OpenAI API bills are hard to understand not because the unit prices are secret, but because one request often triggers several billing dimensions at the same time: ordinary input, cache hits, output, reasoning tokens, images, long-context multipliers, and batch discounts. If you only look at "price per million tokens," it is easy to underestimate the real cost.
This article breaks the subject down in five steps: billing units, the three main accounts, additional rules, an estimation formula, and the order in which to cut costs. All unit prices, discounts, and thresholds are subject to model pricing and OpenAI's official documentation. This article does not hard-code any numbers.
Note: OpenAI, GPT, and ChatGPT are trademarks of their respective owners. This article is provided for technical explanation only. Prices vary by model and billing period; before you build a budget, check the on-site model pricing.
1. First, agree on the unit: a token is not a character
A token is the smallest unit of text a model processes. A single Chinese character, an English word, or a punctuation mark may correspond to one or more tokens. Images, tool definitions, and structured schemas are also converted into tokens. When budgeting, do not estimate with "word count × unit price." Use the usage field returned by the API instead.
The input of a request usually includes:
- the system prompt and developer instructions;
- previous messages in the conversation;
- tool (function) definitions and parameter schemas;
- retrieved documents, web page text, and code snippets;
- tokens converted from images, audio, and other non-text inputs.
The output covers the text the model generates. For reasoning models, internal reasoning tokens are also billed at the output rate, even when the interface does not show the full thought process.
| Billing item | Counted in the bill? | Notes |
|---|---|---|
| Regular input tokens | Yes | System prompts, message history, and tool definitions all count |
| Cached input | Yes, at the cache rate | Only stable prefixes are likely to hit |
| Output tokens | Yes | Usually priced higher than input |
| Reasoning tokens | Yes, at the output rate | Common in reasoning models; the full text may not be shown |
| Image input | Yes | Converted into tokens based on image size and detail level |
| Failed requests | Depends | Most 400-class errors are not billed; partial output may be |
2. Input, output, and cache: the three main accounts
The input account asks what every request carries. Many teams lose money not on the question itself but on resending the whole document, the full set of tool definitions, and the entire message history on every call. However low the input unit price is, it becomes the main cost once multiplied by request volume.
The output account asks how much the model writes. Output is usually priced higher than input, and long reports, long code, and verbose explanations all amplify cost. Setting a reasonable max_tokens ceiling and asking for structured output is more effective than compressing the answer afterward.
The cache account asks what gets reused. OpenAI's prompt caching matches on prefixes: only when the beginning of a request is exactly identical can later content hit the cache. Put the fixed system prompt, tool definitions, and knowledge-base instructions first, and the changing user question last; only then does the hit rate become stable. Caching has a minimum prefix length and a lifetime; it is not permanent storage, and enabling it does not mean every request will hit.
A common misjudgment is to treat caching as "the second request is free." In reality, the cached portion is billed at a lower cache-read price, while the uncached portion is still billed at the regular input price. On the bill, look at cached_tokens and regular input tokens separately.
3. Additional rules: long context, batch, and reasoning models
Beyond the three main accounts, several rules change the final amount.
| Rule | Trigger | Effect on the bill |
|---|---|---|
| Long-context multiplier | A single input exceeds the model's threshold | The entire request is billed at a higher multiplier, not just the excess portion |
| Batch processing | Using the Batch API and accepting asynchronous results | Usually has a separate discount; suits offline jobs |
| Reasoning effort | Raising reasoning effort | More reasoning tokens; higher output cost |
| Image detail level | Using high-detail image input | More tokens are converted from the image |
| Retries | Resending after a 429 or 5xx | Every retry is a new request and may be billed again |
Take the on-site GPT-5.5 documentation as an example: when a single input exceeds 272K tokens, the entire request triggers input and output multipliers. Thresholds like this belong in your budget model, not in a surprise at the end of the month.
4. How to estimate the cost of one request
Treat the unit prices as variables and use this formula:
Cost per request =
uncached input tokens × input unit price
+ cached tokens × cache-read unit price
+ output tokens (including reasoning tokens) × output unit price
+ image/tool and other additional tokens × their unit prices
Final amount = cost per request × long-context multiplier × batch discount
For a monthly budget, use last week's real logs to calculate the share of each token type, then multiply by the unit prices. Do not work backward from an "average cost per call," because the distribution of long and short requests varies widely and an average hides anomalies.
5. Which fields to log for usage
Responses from OpenAI-compatible APIs usually include a usage object. Put the following fields into your logs; reconciliation will take far less time:
import json
from openai import OpenAI
client = OpenAI(
api_key="sk-your-key",
base_url="https://api.zhongshangqihang.com/v1",
)
resp = client.chat.completions.create(
model="gpt-5.5",
messages=[{"role": "user", "content": "Summarize the risks in this contract in three sentences."}],
max_tokens=800,
)
u = resp.usage
prompt_details = getattr(u, "prompt_tokens_details", None)
completion_details = getattr(u, "completion_tokens_details", None)
record = {
"request_id": resp.id,
"model": resp.model,
"prompt_tokens": u.prompt_tokens,
"cached_tokens": getattr(prompt_details, "cached_tokens", 0) or 0,
"completion_tokens": u.completion_tokens,
"reasoning_tokens": getattr(completion_details, "reasoning_tokens", 0) or 0,
"total_tokens": u.total_tokens,
}
print(json.dumps(record, ensure_ascii=False))
If a gateway or SDK version does not return one of these detail fields, fall back with getattr so that logging code never interrupts the main flow.
6. The order for cutting costs: do the free things first
Ranked by effort versus payoff, work through these in order:
- Measure first: break down input, output, cache, and retry tokens by business line, and find the largest item.
- Shrink the input: use retrieval, summaries, and chunking instead of pasting whole documents, and avoid triggering long-context multipliers.
- Stabilize the prefix: put the system prompt, tool definitions, and fixed knowledge base first to reduce cache misses.
- Limit the output: set
max_tokensand require JSON or fixed fields to cut uselessly long answers. - Route by task: send simple classification, extraction, and rewriting to lightweight models; reserve flagship models for complex reasoning.
- Use batch processing: send offline jobs through Batch and accept higher completion latency in exchange for a discount.
- Set budgets: set quota limits and alerts per project and per key so abnormal usage can be stopped quickly.
- Review retries: use exponential backoff for 429 and 5xx, set a maximum retry count, and avoid duplicate billing.
7. FAQ
Why is output more expensive than input?
Generating each token requires step-by-step computation, which costs more compute than reading existing context, so output is usually priced higher. Long answers, long code, and reasoning traces all amplify this part of the bill.
Does caching happen automatically?
Most OpenAI-compatible APIs attempt prefix caching automatically, but a hit depends on whether the prefix is exactly identical, whether it reaches the minimum length, and whether the cache is still alive. A stable prefix is the prerequisite for a high hit rate.
Are reasoning traces billed?
Yes. Reasoning tokens are usually billed at the output token rate, even if the client cannot see the full reasoning content. Raising reasoning effort increases both latency and cost.
Why are two identical requests charged differently?
Input length, output length, cache hits, model snapshot, and retry count may all differ. Align them with the request ID and usage logs to locate the difference.
Is gateway pricing the same as OpenAI's official pricing?
Not necessarily. A gateway may include exchange-rate, payment-channel, and service costs, and may also run promotions. Check the on-site model pricing; do not budget from an old screenshot.
Next steps
Start by registering for the ZSQH AI Gateway, run a minimal request with a test key, connect your usage logging, and then split the budget by business line. Models, context limits, and prices are subject to model pricing.
Related reading:
