The Real Cost of Using AI APIs: Tokens, Caching, Tools, and Retries

Token prices are unit rates, not the cost of a useful outcome. The number that matters operationally is total spend divided by accepted tasks, with failed requests, retries, tool charges, and reviewer time visible. A model with a lower input rate can cost more if it produces long output or needs repeated repair. A premium model can cost less per accepted result if it finishes reliably with fewer tokens—but that must be measured on your workload.

For a basic text request, estimated model cost equals uncached input tokens times the input rate, plus cached input tokens times the cached-input rate, plus output tokens times the output rate. Divide each token count by one million when prices are quoted per million. Add cache-write charges, tool-call charges, storage, and any platform fees separately.

Using the reviewed GPT-6 Astra Standard rates, 20,000 uncached input tokens and 4,000 output tokens estimate to $0.40: $0.20 input plus $0.20 output. Ten identical successful tasks estimate to $4 before other charges. If only eight are accepted, the model spend per accepted task becomes $0.50 even before reviewer time.

At the review date, the Astra model page lists cached input at one tenth of its uncached input rate, cache writes at 1.25 times uncached input, Batch and Flex at half of Standard, and Fast mode at twice the applicable rate. It also lists higher full-request rates when input exceeds 272,000 tokens. These modifiers apply under specific product conditions; do not subtract a cache discount until your logs confirm a cache hit.

Agentic workflows may call search, computer-use, image, or other tools that have separate prices. Timeouts and cancelled jobs can still consume tokens. Large repository snapshots increase input repeatedly unless the harness reuses stable prefixes effectively. Human review, incident response, privacy assessment, and vendor switching are also real costs even though they do not appear on the model invoice.

• Log tokens, cache status, tool calls, latency, retry reason, and acceptance result per task.
• Set a maximum attempts policy so a failing workflow does not loop indefinitely.
• Separate development experiments from production usage in budgets and dashboards.
• Recalculate when the provider changes prices, model aliases, or context thresholds.

Use a per-request ceiling to catch runaway context, a per-task ceiling to include retries, and a monthly ceiling to protect the account. Alert before the hard limit. For batchable, non-urgent work, compare discounted asynchronous modes with the operational cost of delayed results. For urgent work, measure whether faster service reduces downstream labor enough to justify the premium.

Did you count output at its own rate? Are cache writes and confirmed hits separated? Does the request cross a long-context pricing threshold? Are tool calls and failed retries included? Is the denominator accepted tasks rather than requests? Does the budget include reviewer minutes and a stop limit?

• Did you count output at its own rate?
• Are cache writes and confirmed hits separated?
• Does the request cross a long-context pricing threshold?
• Are tool calls and failed retries included?
• Is the denominator accepted tasks rather than requests?
• Does the budget include reviewer minutes and a stop limit?