5 MIN READ
Picture a customer support bot in production. Someone asks a question. The bot answers, confidently, in perfect English, with a citation to a 2024 paper and a footnote to a real journal. Half of it is invented. The customer does not know. The bot does not know. Nobody shipping the bot knew, until the support tickets started coming in.
AI does not flag its own mistakes. There is no internal confidence signal, no hesitation, no pause before a wrong answer. The work of catching the failure has to happen somewhere downstream, and production teams in 2026 are doing that work against four predictable failure modes.
Failure mode 1: the fabricated citation
Ask the model for a citation. It hands back a paper title, a URL, sometimes a DOI. Roughly a third of the time, none of them exist. The citation gets surfaced to the user inside an answer, a report, a research summary. If they click through, they find a 404, a parked domain, or in the worst case a cloned page that hands them a phishing payload. They click because the format looks right. There is no reason not to.
The fix lives in the application layer, not the model. Before any citation reaches the end user, the application fetches the URL, checks that the response is real, checks that the response actually says what the model claimed, and either confirms the citation or strips it. The fetch costs a few hundred milliseconds per citation, and the wiring is work that has to ship with every release.
Failure mode 2: the wrong number
This one is more dangerous than the fabricated citation, because the number looks almost right. The model quotes a 7.4 percent figure where the real answer was 4.2 percent, attributes a quote to a paper that did not say it, or gets a date wrong by a year. The number ends up in a slide, a paper, a press release, a regulatory filing. A careful reader catches it later. The retraction is what gets remembered.
Number grounding is the standard fix. Pass the source data into the prompt, force the model to cite the row, then have the application verify that the number in the response matches the number in the source. None of that is free. The source data has to be cleaned, current, and reachable from the model in the first place, which is usually work the data team is already doing poorly.
Failure mode 3: the invented API
Most agent features break this way first. The model decides to call an endpoint that does not exist, gets a parameter wrong, or uses the wrong signature for a function that has been deprecated. The function call fails. The person using the feature gets a generic error and assumes the AI is broken. Often they are right.
Schema validation closes most of this. The application checks every function call against the live API schema before the call goes out, stubs the test environment with a known good response, and degrades gracefully when the schema is wrong. None of that ships in any agent framework out of the box. It is the kind of work that quietly decides whether an AI feature works in production or just looks good in a demo.
Failure mode 4: the wrong answer to a hard question
The hard ones. The model answers a serious question, in serious prose, with the kind of confidence that reads as expertise. The answer is wrong. The user does not know it is wrong. A decision gets made on the wrong answer, and the consequences arrive months later when the patient is misdiagnosed, the legal brief cites a case that does not exist, the deployment plan commits to a budget that does not work.
Evals catch the obvious stuff. The made up URLs, the invented APIs, the numbers that are clearly off. Less clear are the subtle cases, the ones that look right, read right, and pass every automated check. The rough sense inside most evaluation teams is that the obvious failures are most of what evals catch today, and the long tail of subtle failures is where the real risk lives. Evals are getting better. They are not going to get to 100 percent, and the application has to assume the rest.
What the production team has to do
Treat the model output as untrusted input. Every citation, every number, every function call, every answer gets validated downstream before it reaches the user. Most of it is plumbing nobody puts in a launch announcement, and all of it counts as the work that turns a demo into a product.
Build human review into the workflows where the failure actually matters. A wrong answer in a marketing email is annoying. In a clinical summary it is a lawsuit. The expensive step is the right step for the workflows where the cost of being wrong is real, and the workflows where the cost is real are not most of them.
Instrument the AI features the same way the rest of the system is instrumented. Input, output, latency, cost, failure rate, human override rate, user feedback, all of it logged somewhere anyone can actually query. Without that, nobody knows which hallucination just happened or why, and the next release ships with the same bug.
AI is going to keep hallucinating. The interesting question is what the application does about it, and the only honest answer is that the application has to assume every response is guilty until proven otherwise. The teams shipping this well in 2026 built the guardrails before the AI feature went into production, not after the first public failure.
The bottom line
Validation, grounding, schema checks, human review on the high stakes workflows, observability that actually works. None of it is glamorous. All of it is the difference between an AI feature that survives contact with a real user and one that ends up as a cautionary thread on Hacker News.
Sources & Further Reading
All claims in this article are sourced from primary documentation, vendor advisories, and reputable security researchers.
Spotted an error? Email the editor. Corrections are issued with a visible correction note.
Editorial standards. Every article on humanrequired.org is reviewed by a human editor before publication. AI may assist with drafting or research; final editorial control is human. Read the full standards.
