The working vocabulary / Nate Hamilton
Lab Notes: the terms for running a company with AI agents
The working vocabulary: terms I actually use to run a company on AI agents, defined plainly.
Last updated
01 / Lab Notes
Agent vs assistant
With an assistant, I use the conversation to help finish a task myself. An agent takes responsibility for a defined area of work under a charter and brings back evidence of what happened. I distinguish them by who carries out the work, whatever the tool is called. The human remains accountable for what is accepted as done.
Why it matters. I choose conversational help when I want to think through a task myself. Handing over the work requires a job with clear limits, rules for what the agent may do, and a check before accepting it.
The comparison: agents, assistants, and an agent-run company
Back to index ↑02 / Lab Notes
AI-agent-run company
An AI-agent-run company is a business where a human sets direction and AI agents execute work within defined responsibilities. Each agent owns a domain, carries work through its permitted steps, and returns an outcome that can be checked. Accountability for the company’s choices and accepted work stays with the human.
Why it matters. I use the term to describe who carries out the work. Human judgment remains part of it. Charters, handoffs, and receipts make that division of responsibility concrete.
The comparison: who carries the execution
Back to index ↑03 / Lab Notes
Bound capabilities, not behavior
This design rule limits what an AI can actually do through its tools, permissions, and validation rules. A constraint enforced by code exists independently of whether the model follows an instruction. A prompt describes the intended behavior; the surrounding system enforces the available actions.
Why it matters. I want a boundary I can check, even after the model changes. The limits in an agent’s written job description also need to apply to its tools.
Boundary 13: enforce authority inside tool invocation
Back to index ↑04 / Lab Notes
Bus (message bus)
The shared line the AI agents use to pass work and messages to each other, so a job can move from one agent to the next without a human relaying it.
Why it matters. It's how a team of agents coordinates instead of each one working blind.
Operations: how agents coordinate
Back to index ↑05 / Lab Notes
Charter
The written job you give an AI agent: what it owns, what it's allowed to touch, and when to stop.
Why it matters. A clear charter is the difference between an agent that helps and one that wanders. It's the first thing to write before you hand over any real work.
Delegation: define the job before handing it over
Back to index ↑06 / Lab Notes
Confident wrongness
Confident wrongness is an LLM’s ability to state a false answer with the same fluent certainty as a true one. Tone does not tell you whether the answer is correct. This makes other failures harder to spot: an invented fact, a forgotten constraint, or a missing source can all arrive as a polished answer.
Why it matters. I check the claim against evidence outside the answer. In “I don't let my own AI say a high-stakes release is done. A rival AI signs off first”, even the independent reviewer made findings that needed to be challenged and reproduced.
Failure modes: confident wrongness, symptoms and defenses
Back to index ↑07 / Lab Notes
Context loss (“lost in the middle”)
Context loss is the failure to use information from earlier input because it is no longer available or no longer receives enough attention. Content can drop out when a conversation exceeds its context window. The related “lost in the middle” failure happens even inside that window: material in the middle gets overlooked while the beginning and end still shape the answer.
Why it matters. I split work into chunks with clear limits and keep the checklist outside the conversation. Asking for the exact source passage helps expose missing context; the quoted passage still needs checking.
Failure modes: context loss, symptoms and defenses
Back to index ↑08 / Lab Notes
Decompose-before-dispatch (the 6-item rule)
Decompose-before-dispatch is the practice of splitting work into at most six binary, yes-or-no-verifiable checklist items before assigning it to an agent. Parallel assignments get non-overlapping file boundaries, with at most three agents running in parallel. My dispatch experience led me to a working limit of six. That limit cannot guarantee results for every model or task.
Why it matters. An overloaded agent can omit the bottom of the list and still report completion. I use the original checklist to verify the delivered work. The agent’s summary cannot replace that check.
Operations: bounded work and durable handoffs
Back to index ↑09 / Lab Notes
Director role
The director role is the human responsibility for deciding what to build, setting constraints, and verifying the receipts. Agents handle execution and coordinate work within their charters. I make the calls about purpose, acceptable outcomes, and decisions that exceed an agent’s authority.
Why it matters. My review needs to answer whether the result serves the original purpose. A count of tasks or commits cannot make that decision for me.
The guide: what the human actually does
Back to index ↑10 / Lab Notes
Evidence graph (provenance)
An evidence graph is a structured chain connecting a source document to an extracted finding, an assessment, a recommendation, a human decision, and an outcome. Provenance is the record of where each step came from. The links let a reviewer trace a decision back to the particular source and field that informed it.
Why it matters. In a regulated decision workflow, an AI error has to be traceable to the step that failed. The design keeps the chain as data a reviewer can search, so a review can distinguish incorrectly extracted information from a bad assessment.
Boundary 8: typed decisions and evidence
Back to index ↑11 / Lab Notes
Fit / Stretch / Trap
Fit / Stretch / Trap is a three-question screen for an AI use case: is the task well represented in public writing, is being wrong cheap or checked, and are you thinking with the AI or handing it your judgment? A fit clears the questions and matches a familiar pattern such as drafting or summarizing. A stretch needs deliberate verification; a trap lacks necessary grounding or lets consequential errors pass unchecked. Outsourcing judgment can turn a proposed fit into a stretch or trap.
Why it matters. I use these questions before choosing the first job to hand over. If the model lacks the facts, supplying the necessary context is part of making the task workable.
Delegation: choose a bounded first job
Back to index ↑12 / Lab Notes
Gate (built-right / will-it-earn)
A gate is a decision checkpoint with the authority to stop work from moving forward. In my operating model, the built-right gate judges technical readiness and the separate will-it-earn gate judges whether the work serves a real customer and can earn. Each owner has an independent veto; passing one gate does not satisfy the other.
Why it matters. “We gave two different agents the power to block a release — on purpose.” records why I separated these judgments. Under that rule, finishing a product requires getting it in-market and earning after deployment.
“We gave two different agents the power to block a release — on purpose.”
Back to index ↑13 / Lab Notes
Hallucination
When an AI states something false as if it were fact: an invented number, a made-up source, a policy it was never told.
Why it matters. An authoritative tone is no evidence the answer is correct, which is why the work gets checked at all.
Failure modes: why AI's work needs checking
Back to index ↑14 / Lab Notes
Heartbeat
A heartbeat is a scheduled run that first checks whether work is due and exits when nothing is. When work is due, the agent inspects messages and current state and decides what to advance. The cadence is reduced when a machine is short on resources, and producing nothing can be a valid result.
Why it matters. A logged decision to do nothing can be the correct result. “The smartest thing my night-shift agent did all month was refuse to publish” records a content agent refusing a redundant post and improving existing work instead.
“The smartest thing my night-shift agent did all month was refuse to publish”
Back to index ↑15 / Lab Notes
Instruction drift
Instruction drift is the gradual failure to follow constraints that were set earlier in a conversation. Intervening context can dilute those instructions, and later requests can pull the model toward agreement instead of the original rules. The output can remain fluent while silently dropping part of the assignment.
Why it matters. Before another task, I repeat the constraints that matter. When a workflow runs long, I start a fresh session with a clear brief. Session-level instructions help, but constraints that must hold also need checks outside the model.
Failure modes: instruction drift, symptoms and defenses
Back to index ↑16 / Lab Notes
Jagged intelligence
Jagged intelligence is the uneven capability of an LLM across different kinds of work. It can be strong at pattern-rich tasks such as code generation, extraction, and summarization while remaining inconsistent at real-world grounding, multi-step causal reasoning, or precise numerical computation. Strength on one task does not establish reliability on the next.
Why it matters. I build around what the model does well and use code that follows fixed rules for tasks such as exact calculations. The cost of being wrong sets how much verification the remaining AI work needs.
Failure modes: jagged intelligence, symptoms and defenses
Back to index ↑17 / Lab Notes
LLM (large language model)
The kind of AI behind tools like ChatGPT. It is trained to generate text, which lets it draft, summarize, answer, and follow instructions.
Why it matters. It's fluent and powerful, which is exactly why it can be confidently wrong. Fluent is not the same as correct.
Failure modes: where language models go wrong
Back to index ↑18 / Lab Notes
Receipt (proof-of-work)
A receipt is evidence from the actual result that demonstrates an agent met a specific acceptance criterion. It might be a live URL, a passing probe, or output showing that the required user journey works. A build log or a completion message alone does not prove the live outcome.
Why it matters. I match the receipt to the claim: a page loading proves it can be reached, while a saved change surviving a reload proves it stayed saved. Neither proves that the product earns money.
Operations: prove the work is actually done
Back to index ↑19 / Lab Notes
The sacred test set
The sacred test set is the rule that a representative, quarantined set of cases is reserved for measuring accuracy and regressions: append-only, never used for training, tuning, or examples, and applied to every evaluation run and model upgrade, including the cases that produce embarrassing scores.
Why it matters. Once those cases become training material, the score can measure memorization instead of performance on unseen work. Keeping the measurement cases separate is what keeps weak results visible.
Boundary 7: make evaluation owned infrastructure
Back to index ↑20 / Lab Notes
The trust curve
The trust curve is a graduated path from probation through supervised work to autonomy, earned through verified performance on real cases. It starts with human review of every decision, then reduces review for proven low-risk and routine work while keeping anomalies under review. The operator can lower the trust level again when errors appear.
Why it matters. I want observed accuracy for that kind of decision to determine how much the agent can do on its own. The delegation playbook applies the same principle: inspect the first bounded job before expanding authority.
Delegation: inspect before expanding
Back to index ↑