The demo account never worked a shift

The demo runs for forty minutes in a conference room with good light. A project sponsor, two department heads and an account the vendor set up the day before click through three scenarios, in order, exactly as rehearsed. Every answer lands where it should. Sign-off happens before the coffee goes cold.

Three weeks later the system goes live, and the first person who actually has to use it is not in that room. He is on the floor, he has maybe forty seconds between parts, and he has never seen the script the demo followed. Inside his first shift he types three things the demo account never tried — a part number with a stray letter nobody remembers the reason for, a shortcut the old system taught him years ago, a question asked mid-way through a different one. Two of the three come back wrong.

Nobody in that conference room lied. The demo did exactly what it was built to do. It was simply never run by the person who has to live with it.

Why we build it this way

Most systems are accepted by the people who bought them, not the people who will run them every day. A sponsor and a couple of managers watch a demo, approve what they see, and sign off before the floor has touched anything. That is standard practice, and it is also where a lot of industrial AI projects quietly go wrong.

The problem is not that the demo was dishonest. It is that the demo account behaves nothing like the real users. It never has forty seconds to decide. It never carries a habit from the system it replaced. It never types the same part number two different ways because two supervisors taught it, years apart. A test script written by the project team only tests what the project team already thought to test — the same blind spot that let the gap into the system in the first place.

A longer scripted test phase does not fix this, because the people writing the extra scripts are still the project team. The fix is harder to agree to.

This is uncomfortable on purpose. Things that looked finished in the demo turn up broken in the first real week, in front of the client, by someone who has no reason to be polite about it. That is the point. Found then, it costs an afternoon. Found after handover, it costs a workaround nobody writes down and a system the floor quietly stops trusting.

How it works, roughly

Before anyone signs off, the actual users run the actual system on the actual floor for long enough to hit an ordinary week, not a rehearsed hour. In practice that means watching for:

• The jobs as they actually arrive, not the scenarios chosen for the demo
• Every shortcut and habit a person already carries in from the system it replaced
• Every point where what the system says and what an experienced person would say disagree

Each disagreement goes back to the person who caught it and whoever is accountable for that part of the work — never to whoever built the system. While we are still on site, we change what needs changing and run the same week again, until the actual users stop finding new ones.