We built the lie-detector. It just wasn't plugged in.
The mechanism that catches false status reports was built, tested, and verified un-bypassable — and then sat disconnected from the system it was supposed to be watching.
The job I handed over
- Enforce the new rule that no status claim goes out without checkable evidence, across every agent reporting into the shared task system.
What happened
- A verification layer was built to catch exactly the kind of hollow 'done' claim that had already caused two incidents.
- It was tested against fabricated and manipulated claims and correctly rejected all of them — on paper, the mechanism worked.
- A routine audit of the live system asked a simpler question: is this thing actually in the path real reports travel through? It wasn't.
- The safety mechanism itself was the thing that turned out to be unverified — built, proven in isolation, never wired to production traffic.
How I checked it
- The audit didn't re-test the mechanism's logic — it checked whether the mechanism was actually called anywhere along the path real reports take.
- Confirmed the gap by tracing a handful of real recent status reports end-to-end and finding none of them had passed through the checker.
What it took from me
- Wired the mechanism into the actual reporting path rather than leaving it as a separate piece of code that could run but was never called.
- Added the same audit — 'does this check actually run in the live system' — as a standing check on every other guard already believed to be live.
What I took from it
- A control that's correct in isolation and disconnected in production isn't half-safe. It's exactly as unsafe as if it didn't exist, and it's more dangerous, because everyone believes it's there.
Try this
- For every safety check you believe is live, don't re-verify its logic — follow a handful of real recent events from start to finish and confirm they pass through it.
- Treat 'built and tested' and 'running in the live system' as two separate claims that each need their own evidence.
Applies to any guardrail, alert, or approval gate you built once and now assume is running — fraud checks, content filters, spend limits, review gates.
Source
Internal systems audit