The price list of the major AI APIs is public. The cost of running a real AI agent in production is not. The total cost of ownership of an AI agent in 2026 is typically 3x to 10x the per call price of the underlying model. Companies that budgeted for the per call price are discovering this, and the discovery runs as the biggest unspoken issue in AI deployment.
Every AI vendor publishes a price list. OpenAI charges $X per million input tokens and $Y per million output tokens. Anthropic charges a similar shape. Google charges a similar shape. The price list is transparent. The price list is also the price for one model call, with no retrieval, no embeddings, no tool calls, no retries, no monitoring, no evaluation, no human in the loop, and no failure recovery. The published price runs as the price for the part of the AI that the vendor sells you. The published price is not the price of the AI agent you actually run.
The real cost of an AI agent, in production, with the features the marketing demo implied, is a multiple of the published price. The multiple is rarely less than 3x. The multiple is often 10x. The multiple is sometimes higher, especially for agents that do significant retrieval or that perform many tool calls per task. The companies that budgeted for the published price are discovering the multiple the hard way.
The published API price
The published price is for a single model call, with a defined input and a defined output. The call has no context beyond the input. The call has no memory of previous calls. The call has no tools. The call has no retrieval. The call has no safety guardrails beyond what the model itself does. The call is, in other words, the simplest possible interaction with the model.
For most production AI agents, the published price amounts to the smallest part of the bill. The largest part of the bill is everything around the published call. The everything is, depending on the architecture, retrieval, embeddings, tool calls, retries, evaluation, monitoring, logging, safety, and human in the loop. The everything is also, in most cases, billed at the same per token rates as the model call itself.
What is not in the published price
Eight things are typically missing from the published price. Retrieval, the vector search the agent does to find relevant context, billed at the embedding API rate for the indexed documents, plus the search and reranking cost. Embeddings, the cost of converting the agent’s context into vectors, billed at the embedding API rate, recomputed every time the context changes. Tool calls, the cost of the API calls the agent makes to external systems, billed at those APIs’ rates, plus the model’s cost of generating the tool arguments. Retries, the cost of running the agent again when the first attempt fails or returns a malformed response, billed at the model rate for each retry.
Monitoring, the cost of running an observability platform on the agent’s behaviour, billed at the observability vendor’s per event rate. Evaluation, the cost of running an evaluation suite to check the agent’s output, which often uses another model call to score the output, billed at the evaluator’s rate. Safety, the cost of running the output through a content classifier or a guardrail model, billed at the safety vendor’s rate. Human review, the cost of having a human in the loop for a fraction of the agent’s outputs, billed at the human’s time.
None of these are in the published model price. All of them are real. All of them add up.
The retrieval bill
Retrieval augmented generation, RAG, amounts to the standard pattern for grounding an agent in the customer’s data. The agent takes the user’s query, embeds it, searches a vector database for relevant documents, fetches the documents, and feeds them to the model as context. The pattern is sound. The pattern is also expensive.
The embedding of the query is one call to the embedding API, billed at the embedding rate. The search is free, usually, because the vector database is self hosted. The fetch of the documents is free, in terms of the model, but the documents are then fed into the model as context, billed at the model rate. The bigger the documents, the bigger the context, the bigger the bill. A typical RAG pipeline, with 5 to 10 documents of 1,000 tokens each, adds 5,000 to 10,000 input tokens to every model call. The cost of those tokens, at the published rate, is often more than the cost of the model call itself.
The retrieval bill, in other words, counts as the largest single addition to the published model price. The companies that built a RAG pipeline on the assumption that retrieval was free are discovering that retrieval is, in fact, the dominant cost.
The embedding bill
Embeddings are the most undercounted cost in the AI agent stack. The embedding cost is per token, billed at a lower rate than the model call, but the embedding is computed for every document that might be relevant, plus every chunk of every document, plus every query, plus every re-embedding when the documents change. A document corpus of 10 million tokens, re-embedded every time the corpus changes, costs $30 to $100 per re-embed at current embedding API rates. A document corpus that changes daily, which is most production corpora, costs $10,000 to $30,000 per year in embedding calls alone.
The embedding bill is also a bill that is rarely surfaced in the AI agent budget. The embedding is a separate API. The embedding is on a separate billing relationship. The embedding is, in many cases, not even on the same provider as the model. The total cost of the embedding, across the year, is often larger than the cost of the model itself, for an agent that does a lot of retrieval.
The retry and failure bill
AI agents fail. The model returns malformed JSON. The tool call returns an error. The vector search returns nothing relevant. The agent loops, calls the same tool 5 times, gets 5 errors, gives up. The user, frustrated, retries the query. The retry cost is, in the worst case, a multiple of the original cost, because the retry often involves more context, more tool calls, and a more elaborate prompt to the model.
The retry rate for a well engineered AI agent, in 2026, is typically 5 to 15 percent. The retry rate for a poorly engineered AI agent, or for an agent that handles complex edge cases, can be 30 percent or more. The retry cost counts as the published model price, multiplied by 1.05 to 1.30, every time the agent runs. The retry cost is rarely budgeted, because the retry is, by definition, the cost of a failure, and the budget was made for the success case.
The evaluation and monitoring bill
An AI agent in production is not a one time model call. The agent is a system that needs to be evaluated, monitored, debugged, and improved. The evaluation suite is a set of test cases, run periodically, that check the agent’s behaviour against expected outputs. The monitoring platform is a set of tools that observes the agent’s behaviour in production and alerts on anomalies. The debugging workflow is a set of tools that lets the engineer replay a production interaction with the model and understand what went wrong.
All of these have costs. The evaluation suite uses another model to score the agent’s output, billed at the evaluator’s rate. The monitoring platform bills per event, per trace, per stored log. The debugging workflow bills for storage and for the replay model calls. None of these are in the published model price. All of them are real. The total cost is, in most cases, 20 to 50 percent of the published model price, on top of everything else.
The safety and guardrail bill
AI agents in production need safety controls. The output of the agent is, in most cases, run through a content classifier to check for harmful content. The input to the agent is, in many cases, run through a prompt injection detector to check for attacks. The agent’s tool calls are, in the well engineered cases, run through a policy check to ensure the call is within the agent’s authorised scope.
Each of these is a model call, or a tool call, billed at its own rate. The content classifier is typically a small, fast model, billed at a lower rate. The prompt injection detector is similar. The policy check is a structured lookup, often free. The total safety bill, for a well engineered agent, is 10 to 30 percent of the published model price, on top of everything else.
The real TCO
Add it all up. Published model price, multiplied by 3 to 10, depending on the architecture, the retrieval volume, the retry rate, the safety requirements, the monitoring load. The total cost of ownership of an AI agent, in production, with the features the marketing demo implied, is rarely less than 3x the published model price. The total cost is often 10x. The total cost is sometimes higher, especially for agents that do significant retrieval or that have aggressive safety requirements.
The companies that budgeted for the published price are discovering the total cost. The discovery is, in some cases, forcing them to scale back the agent. The discovery is, in other cases, forcing them to find a cheaper model, which forces them to compromise on quality, which forces them to find other ways to maintain the quality, which adds cost in a different category. The discovery is, in most cases, a budget conversation that the AI team did not want to have and the finance team did not see coming.
What the realistic budget looks like
A realistic AI agent budget, in 2026, allocates for the published model price, multiplied by 5x as a working estimate. The 5x is split as follows. 1x for the published model calls. 1.5x for retrieval and embeddings. 0.5x for retries and tool calls. 0.5x for evaluation and monitoring. 0.5x for safety and guardrails. 0.5x for human review of a fraction of outputs. 0.5x for the engineering team to maintain the agent, run the evaluations, debug the failures, and ship the improvements.
The 5x estimate is conservative for agents that do significant retrieval. The 5x estimate is optimistic for agents that handle complex edge cases or that require aggressive safety. The 5x estimate should be revised upward as the agent is used more, as the corpus grows, as the safety requirements tighten, and as the human review fraction increases.
The bottom line
The published model price becomes the price for one call, in isolation, with no context, no retrieval, no safety, no monitoring. The published model price amounts to the price the vendor sells. The price the buyer pays sits as the published price multiplied by 3 to 10, depending on the architecture. The companies that budgeted for the published price are discovering this. The companies that budgeted for the multiple are running agents that are not as cheap as the marketing implied, but are at least budgetable.
The fix is not to be cheaper. The fix is to be honest. The published price is real, the multiple is real, and the gap between them counts as the conversation the AI industry has been avoiding. The companies that have the conversation early, that budget for the multiple, that design the agent to minimise the multiplication, will deploy AI in production at a cost the CFO can sign off on. The companies that avoid the conversation will discover the multiple in production, at scale, at the worst possible time.
Sources & Further Reading
All claims in this article are sourced from primary documentation, vendor advisories, and reputable security researchers.
Spotted an error? Email the editor. Corrections are issued with a visible correction note.
Editorial standards. Every article on humanrequired.org is reviewed by a human editor before publication. AI may assist with drafting or research; final editorial control is human. Read the full standards.
