Summary

Tokens are subword pieces, so character and word counts lie, and they lie worst on code, structured data, and languages the tokenizer saw less of. Count with the real tokenizer before you send, include message overhead, tool definitions, schemas, and images, and verify your count against the provider's reported usage so drift is visible. Token counts do not transfer between providers, so counting belongs in the adapter alongside every other provider difference.

Every request must satisfy system plus history plus retrieved context plus output reserve within the window, and the output reserve is the term people forget. Check the budget before sending, carry the counts on the exception, and choose a truncation strategy deliberately: dropping oldest is free and forgets silently, importance ranking retains better, and summarisation retains best at the cost of a model call. Compact on a proportion of budget rather than a turn count, and compact early rather than at the limit. When input genuinely does not fit, reject, route, split, or retrieve, and say which you did.

Output costs several times input, cached input costs a fraction, and reasoning tokens are billed as output while remaining invisible. Record cost per request, session, user, and feature, and use context variables to attribute spend without threading parameters through every function. Keep prices as dated data rather than constants so history stays correct, and reconcile against the invoice monthly, where a large gap usually means a whole category of requests is unattributed.

Control spend before the call, not after: per-user quotas, a global cap, and a kill switch that works without a deployment. Structure prompts so stable content comes first, because caching matches a byte-identical prefix and one early timestamp costs everything after it. Route cheap first and escalate on a hard signal, but measure the escalation rate, since a high one costs more than going direct. Move latency-tolerant work to batch pricing. Reduce output rather than input where you can, since output is both the expensive half and the slow half. And put cost ceilings in CI, asserted on tokens rather than dollars.

Key takeaways

• Characters divided by four is wrong, and it is wrong in the dangerous direction for code and non-English text
• Count tools, schemas, and images, which are the three things people omit
• Token counts do not transfer between providers
• Reserve space for the output, or the request fits and the answer does not
• Reasoning tokens are billed as output and consume window while staying invisible
• Truncation that forgets silently is worse than truncation that says so
• Compact at seventy percent of budget, not at the limit
• Output costs several times input, so output length is the dominant lever
• Attribution is what turns a bill into a decision
• Price entries need effective dates, or last quarter's costs change when a price does
• A thirty percent reconciliation gap means a category of requests is unrecorded
• Check budget guards before the call, since checking afterwards is a report
• One volatile value early in a prompt destroys the whole cache
• Escalation bills both calls, so measure the rate before assuming it saves money
• Assert token ceilings rather than dollar ceilings in CI

Common mistakes to remember

• Estimating tokens by dividing characters by four
• Counting message content and omitting tool definitions
• Constructing a tokenizer encoding on every call
• Using one provider's token count to price another
• Omitting the output reserve from the budget
• Truncating input silently and answering from part of it
• Dropping the system message or the current question when truncating
• Reordering turns when importance ranking and confusing the sequence
• Summarising a summary repeatedly until the context is meaningless
• Using break instead of continue when fitting chunks to a budget
• Using float for money
• Counting only successful requests in cost records
• Hardcoding prices, or defaulting an unknown model to zero
• Adding cached tokens to the input total instead of subtracting them
• Checking a spend cap after the request rather than before
• Putting a timestamp at the top of the system prompt
• Serialising tool definitions in a non-deterministic order
• Assuming routing saves money without measuring the escalation rate
• Running latency-tolerant batch work at real-time prices
• Asserting dollar ceilings in CI that fail when a provider changes prices