A green checkmark said 4 people were reachable. The real number was 250.
The agent hadn't broken the rule. It had applied the rule to half the data — and every step after that looked correct.
The job I handed over
- Build the list of people we can email for a campaign and apply our firm consent rule: nobody gets mailed unless their marketing consent is current.
What happened
- The step reported done: 4 sendable contacts out of roughly 3,100.
- It also warned that 8,314 contacts would be blocked across the whole database.
- It cited the consent rule it had applied and spot-checked samples to prove it.
- It looked rigorous. Nothing in the output suggested a problem with the work itself.
How I checked it
- A separate agent owns one job: when work is marked complete, question the starting assumption, not just the steps.
- The assumption it questioned: is checking consent in only one system even correct for these contacts?
- It isn't. Consent lives in two systems — the current ecommerce platform and the legacy email platform. The step had validated against the ecommerce platform only, silently discarding everyone whose consent came from the legacy email platform.
- A cross-check pulled records from both systems (~24k profiles each) and merged them. About 250 people were reachable, not 4; after 134 cross-source consent conflicts were excluded, 228 were sendable.
- The alarming 8,314 blocked figure came from looking at only one system. The underlying data was fine.
- The actual defect: an unnecessary repeat check against only the ecommerce platform buried in the tool that builds the audience, crushing a 250-person audience to 4 on every run.
What it took from me
- Fixed the tool that builds the audience to use merged consent from the ecommerce platform OR the legacy email platform.
- Fixed the ~134 genuine conflicts between the systems — people who really had opted out in one system.
- Changed how the systems share updates so the two sources stop overwriting each other.
- Re-ran a three-way pull (ecommerce platform + legacy email platform + our database, ~24k contacts) to confirm conflicts went 134 → 0.
- My own time went almost entirely into the decision, not the repair: deciding that merged consent was the correct rule.
What I took from it
- A confident checkmark sitting on a plausible-but-wrong assumption is the most dangerous kind of done. Re-running the same logic would have returned the same green forever.
Try this
- Before you delegate a recurring job, write down how you will check the result first — the specific evidence you'd accept as proof it worked.
- Then write down one assumption the check itself should question. Usually it's "does this source have all the information?"
- Decide in advance what happens when the check fails: who is told, what stops, and what does not get sent.
- Give the check to something other than the worker. A worker that grades itself will grade its own assumptions too.
Applies to any job where the same fact lives in more than one system — customers, consent, inventory, availability, pricing.