463 passing tests. A blank website. Both true at once.
Every test was green. Every page was empty. Here's how both can be true — and the one check that would have caught it.
The record
Lab Notes
What the lab tried, what happened, the terms it uses, and the evidence a reader can check.
Real examples
What happened when AI did the work.
What I handed over, what came back, how I checked it — and the part you can try in your own business.
Real cases from the lab's businesses.
The constitution I gave my AI company
An AI company that runs on agents needs rules they can't argue their way out of. So I wrote them down, and I am wiring each one to a gate, because a rule an agent can ignore isn't a rule.
The shared messaging system went dark for twelve hours. We lost nothing, but the requests waiting on me looked ignored.
A dead notifier doesn't drop messages. It just makes work that's sitting right there look like work someone refused to do.
I don't let my own AI say a high-stakes release is done. A rival AI signs off first
The builder said ready. A different model family — a different line of AI models — disagreed. The useful part was making both of them prove it.
We gave two different agents the power to block a release — on purpose.
One agent decides if it's built right. A different one decides if anyone will pay for it. Neither can call it 'done' alone. That split is the whole point.
A green checkmark that isn't the AI's own word: building a quality check with teeth
My quality-check tool failed all 24 controls on a live app. Every failure was in the tool. A confident red can be just as fictional as a confident green.
I put my AI staff on a payroll page — with real commit counts and blanks where I don't measure yet
The commit counts come from real work. The salary column says '—.' I don't measure it yet, and a plausible number would make the page less honest.
1,000 commits looked lost. They were staged in a pull request nobody had merged for six days.
The scary number wasn't lost work — it was work waiting for a human to merge it, which nobody did. The fix was a check, not a rebuild.
A green checkmark said 4 people were reachable. The real number was 250.
The agent hadn't broken the rule. It had applied the rule to half the data — and every step after that looked correct.
98 commits in a week. The commit count tells me almost nothing.
Proof that software work can be handed over. Not proof that the software works.
Three ways to fake a receipt. All three got caught.
Once 'show me the receipt' became the standard, the obvious next question was: what happens when someone tries to forge one?
21 identical blocked messages, for a credential that never existed
An agent invented a requirement, decided it could not proceed without it, and reported the exact same status update twenty-one times before anyone asked what it actually was.
The rule wasn't 'don't lie to me.' It was 'don't lie to any agent, including yourself.'
A blank page reported as done and a safety check reported as armed both traced back to the same thing: a claim nobody could check. That became the one absolute rule.
The smartest thing my night-shift agent did all month was refuse to publish
My content agent turned down a sixth same-day templated post and spent the night checking what was already live. That's a better night's work than another URL.
Three months. ~2,985 commits. 88–100% internal scores. Zero paying users.
Every gate the agent controlled turned green. The one gate that mattered — a real person using the product — never got touched.
We built the lie-detector. It just wasn't plugged in.
The mechanism that catches false status reports was built, tested, and verified un-bypassable — and then sat disconnected from the system it was supposed to be watching.
250 subagents, 300 uncommitted files, and a 92% wrong answer
Handing work to more agents without a limit looked like speed. It was 250 agents editing the same shared files and grading the wrong source of truth, together, at once.
Terms
Lab Notes: the terms
The working vocabulary, with definitions and links to the accounts and guides.
Read the terms →
Evidence
All evidence →commit ·
The merged-consent fix behind the 4-versus-250 audience
The audience builder re-checked consent against the ecommerce platform alone on top of a customer-data flag that already merged both platforms. The double check discarded about 250 genuinely opted-in contacts and left a sendable cohort of 4. The fix replaced the redundant check with a merged-consent rule and added a conflict check that holds back contacts whose two sources disagree.
Read evidence →
doctrine ·
The commit that made a claim without evidence a lie
The rule entered the kernel every agent loads at the start of a session. A hollow claim without checkable evidence is treated as a lie, to Nate or to any agent. The commit records that the rule was verified to load in a headless session.
Read evidence →
commit ·
The receipt gate that checks the claim, not just the signature
The first receipt gate used a wrapper to record the command, exit code, output digest and assertions, then sign them. A validator checked the signature. For a push claim, it re-derived the result from the remote instead of trusting the receipt. Only the push claim had a re-check at the time. The other claim types were marked as stubs.
Read evidence →
log ·
The dead-man alert and the watchdog ledger behind the twelve-hour outage
The notifier processes on one machine stopped checking in. The fleet dead-man fired after 727 minutes. At 16:34 UTC, the watchdog ledger showed four agents on that machine as STALE/DEAD, with ages of 43,677 and 43,643 seconds. Every message stayed on disk and was recovered by reading the log directly.
Read evidence →
doctrine ·
The doctrine commit that split built-right from will-it-earn
The commit adds the product-oversight gates: an intent gate before building and a launch gate before shipping, each with its own veto, held by a different agent from the one that judges whether the work is built right.
Read evidence →
log ·
The content agent's log entry for the night it declined a sixth post
The content agent declined a sixth same-day templated post on August 3. Four posts had gone out that day and one the day before. The content covered 16 trade verticals. The site had 76 published posts with an average quality score of 80 out of 100 and no failures. That night was a research and health-check pass with no code changes. The agent checked keyword-demand data and read the one existing AEO post in full. It decided a second post on that topic would cannibalize the first. On an earlier night, it chose a pricing-accuracy fix on published posts over a new post.
Read evidence →