AI guide / Nate Hamilton
How AI Agents Fail: The Failure Modes and How They Compound
How confident-but-wrong output, drifting from instructions, lost context, and uneven capability cause agent work to fail, and how small failures chain into big ones.
Last updated
Why name the failure?
Name what failed so you can check it
AI agents fail when they invent facts, stop following instructions, or miss information they need. Their fluent delivery can hide those errors long enough for someone, or another step in the workflow, to act on them. “The AI was wrong” gives me very little to fix. Naming what failed tells me where to put the check.
This guide groups model failures into four modes: hallucination, instruction drift, context loss, and confident wrongness. I also include jagged intelligence here because uneven capability changes where I trust the work. The glossary gives the short definitions; this page connects them to symptoms and defenses. This practical framework does not claim to be an exhaustive list of agent-system failures.
01 / Failure modes
Confident wrongness
Confident wrongness is a false answer delivered with the same fluent certainty as a true one. Tone cannot tell you whether it is correct. It makes the other failures harder to see: invented facts, dropped instructions, and missing context can all arrive as polished work.
How it shows up. An agent declares the work complete, or a reviewer declares it broken, without evidence that supports the verdict. I can be persuaded by either if I mistake a clear explanation for a checked result.
How I defend. I check claims against evidence outside the answer. When the result matters, I require evidence before accepting the work. Asking the agent whether it is sure is not an independent check; inspecting the source or reproducing the outcome is.
Lab Notes: Confident wrongness
In “A green checkmark that isn't the AI's own word: building a quality check with teeth”, the checker produced false failures and some checks were not running in the live workflow. I need to verify the checker as well as the agent’s work, and keep unchecked work visibly unchecked.
02 / Failure modes
Hallucination
Hallucination is plausible output presented as fact when it is not true. An agent can invent a citation, a number, or a software interface that looks like the real thing. Confident wrongness describes how convincing that error can sound; hallucination describes the unsupported content.
How it shows up. A reference has a believable title but cannot be found. Generated code calls an interface that does not exist. A summary includes a specific claim that the source never made. A specific claim gives me a detail I can check before trusting it.
How I defend. I give the agent the relevant source material and verify the factual claims against it. I check that a reference exists and supports the claim, and that generated code works on the actual path it will run. Supplying sources helps; it does not make the answer automatically supported by those sources.
03 / Failure modes
Instruction drift
Instruction drift is the gradual failure to follow constraints set earlier in a conversation. Later context can dilute the original instructions or pull the model toward agreement. The answer can remain fluent while the work stops satisfying the assignment.
How it shows up. The first response respects the rules, but a later one drops a constraint. A request for skeptical review turns into agreement, or only some parts of a multi-part assignment survive. I compare the work with the original brief to see what disappeared.
How I defend. I restate important constraints before the next task and restart long workflows with a clear brief. Session-level instructions can help, but rules that must hold also need checks outside the model. I keep the acceptance checklist available and verify the result against it.
04 / Failure modes
Context loss: “lost in the middle”
Context loss is a failure to use earlier information because it is unavailable or overlooked. Content can drop out as a conversation exceeds its context window, the amount of information the model can use at once. The related “lost in the middle” problem can happen inside the window: material in the middle receives less attention than the beginning and end.
How it shows up. An answer contradicts an earlier agreement, ignores a supplied passage, or summarizes the ends of a document while missing a condition in the middle. A plausible account of what we agreed is not proof that the relevant context survived.
How I defend. I split long documents and assignments into chunks with clear limits, preserve the checklist outside the conversation, and start fresh with a checked summary when needed. I ask for the exact source passage and compare it with the original. A missing or invented quote is a reason to investigate; it does not by itself prove which failure caused it.
05 / Failure modes
Jagged intelligence
Jagged intelligence is uneven capability across tasks. A large language model (LLM) can be strong at work with familiar patterns, such as writing code, extracting information, and summarizing. It can still be inconsistent at basing answers on real-world facts, reasoning through cause and effect across several steps, or calculating exact numbers. Success on one task does not establish reliability on the next.
How it shows up. A strong draft earns the agent trust, and that trust carries into a calculation or judgment that needs a different check. The work looks continuous to me, but the required capabilities have changed.
How I defend. I assess what this task needs even when the agent has impressed me on other work. I use code that follows fixed rules for exact calculations and match the checks to the cost of being wrong. Where a rule must be enforced, the surrounding system needs to enforce it regardless of the model’s recommendation.
06 / Failure modes
How small failures compound
Failures compound when an unchecked output becomes the premise for the next step. Hallucination, drift, and context loss can combine; confident wrongness makes the result look trustworthy enough to pass along. Each later step may appear reasonable while depending on an earlier mistake.
How it shows up. Consider a summary that misses a condition in the middle of a document. A later recommendation uses that summary as if it were complete. If it also drops an instruction to verify claims, an invented detail can pass through as fact. The next reader sees a confident recommendation without seeing the gaps beneath it. This is an illustrative chain, not a reported incident or a required order.
How I defend. I check the source and original constraints before the result becomes someone else’s input. I require evidence of the claimed outcome and make missing evidence visible. If a check fails, I revisit the downstream work that depended on that result. A second polished answer alone does not break the chain.
Agent systems also fail outside the model. In “The shared messaging system went dark for 72 minutes. We lost nothing, but the requests waiting on me looked ignored.”, unread messages looked ignored and an escalation timer treated the silence as stalled work. The durable messages survived. That operational failure does not establish model context loss, but it shows how an unchecked interpretation can feed the next action. Read AI Agent Operations: what happens when the system fails for the recovery path.
The examples in AI Guides are my published operating accounts. Their underlying evidence records do not currently have public artifact links; use the notes to understand the checks, then inspect your own result.
07 / Quick answers
Frequently asked questions
What are the main ways AI agents fail?
The four model failure modes are hallucination, instruction drift, context loss, and confident wrongness. Jagged intelligence explains why capability varies across tasks. These names help identify recurring model failures. An agent system can break in other ways too.
Is confident wrongness the same as hallucination?
No. Hallucination is unsupported or invented factual content. Confident wrongness is the convincing delivery that can hide that error, a forgotten instruction, or missing context. I verify the claim against an external source or actual result rather than judging the tone.
Does a larger context window prevent context loss?
A larger window can hold more input, but available text can still be overlooked. Lost in the middle refers to that attention problem inside the window. I keep work bounded and check the exact source passage rather than assuming everything supplied was used.
How do I stop a small agent error becoming a bigger one?
Check the evidence before the output becomes input to another step. Compare it with the source and original constraints, keep unchecked work visibly unchecked, and require proof of the actual outcome. These checks reduce the chance of propagation; they do not guarantee that every error will be caught.