Field Notes
July 24, 20266 min read

Why the Demo Worked and Your Implementation Didn't

Justin Henriksen
Justin Henriksen

Founder & CEO, GetLatest AI

A demo is a film set. Everything in frame looks real. Nothing behind the camera is load-bearing.

Your business is the building that has to stand up in a storm.

A software company I know spent 3 weeks building a lead qualification agent. Demo was genuinely impressive - scored leads, drafted responses, updated the CRM. Team left excited.

3 months later it was running at 60% of what it was supposed to do. Nobody could tell you which 40% had broken. Leads stuck in a queue nobody checked. The agent had drafted 12 responses to the same lead because a webhook fired repeatedly and nobody set up deduplication. The team stopped trusting it and went back to manual - but still technically "ran the agent." Worst of both worlds.

This is predictable. Every failure mode has a name.


Why demos look good

Optimization, not dishonesty. A demo shows peak conditions.

Clean data. One workflow. No conflicting records, no messy import history. The agent sees exactly what it's supposed to see.

That's not your business.

Your business has the same lead submitting your contact form twice - once with a work email, once with a personal email. The agent finds 2 records and has to decide if they're the same person. If you didn't design for that case, it either picks wrong or creates a third record.

Your business has a webhook that went down at midnight. Nobody noticed for 2 days. Zero leads processed. Nothing flagged it. You found out when a prospect emailed asking why nobody followed up.

Your business has an agent drafting responses that cite a pricing tier you retired 4 months ago - because the doc it references hasn't been updated. It had no way to know.

None of these are AI failures. They're normal conditions of a real business. Every operational system you run deals with them. The agent is no different. It just needs to be built like a production system - not just to the point where the demo works.


The work that happens before you touch the technology

The most common reason implementations fail: starting with the tool before understanding the workflow.

Map the actual workflow, not the ideal one. Sit with whoever does the work and watch them do it. The spreadsheet someone opens every time because "the CRM doesn't support that input" - that workaround is load-bearing. The agent inherits every quirk of the process that actually exists. Not the one you wish existed.

Know what the agent needs to touch. Most growth workflows span 4-5 tools. The agent needs credentials for each. Some have official APIs. Some have third-party connectors with their own rate limits. Some have no API and require workarounds. Know which category each system falls into before you design anything.

Understand rate limits. Every API has a ceiling. Agent processes 500 leads in an hour, your CRM allows 100 calls per minute - you'll hit the ceiling. If you haven't designed for graceful handling - slowing down, queuing, alerting a human - the agent fails silently.

Know who owns the credentials. API keys need to live somewhere secure, get rotated when they expire, be owned by a specific person. "The team" is not an owner.

This is the unglamorous work that determines whether everything else holds up.


The design decisions that matter more than which model you pick

What the agent is told. Instructions encode your business rules - who counts as a qualified lead, what tone to use, which deal stages require human escalation, what to do when data is missing. Some rules are hard constraints. Some are soft guidance. Knowing which is which separates an agent that behaves like your best employee from one that drifts in small hard-to-notice ways.

Identity resolution. Same name, slightly different email - "jsmith@company.com" and "john.smith@company.com." Match confidently, flag for review, or treat as new? There's no universally right answer. Only the one that fits your data and your risk tolerance. You need a defined strategy before launch, not after the third duplicate.

What actions the agent is allowed to take. Define these narrowly. An agent that qualifies leads should not have access to delete records. An agent that drafts outreach should not have access to publish to your website. This limits the blast radius when something unexpected happens. And something unexpected will always eventually happen.

How humans stay in the loop. The approval interface matters more than most builders expect. If reviewing the agent's work is slow or awkward, people start approving without reading - or stop using it entirely. The interface needs to show: input the agent saw, reasoning it applied, proposed action. Approve, edit, or reject in seconds.


What makes it hold up over time

A system that works when everything goes right is not a reliable system.

Logging. You need to answer "what did the agent do, and why?" for any action it took. Logs should capture input, reasoning steps, and output - in plain enough language that a non-engineer can read a log entry and understand what happened.

Error handling. When data is missing, an API call fails, or input arrives malformed - each case needs an explicit handling path. "Fail loudly and alert a human" is almost always the right default. "Try again silently and then silently stop" is how you end up with a 3-day gap in lead processing that nobody noticed.

An owner. This is where most implementations quietly die. Agent gets built. It works. Everyone moves on. Nobody is specifically responsible for monitoring it. 3 months later it's partially working, nobody knows which parts, and diagnosing what broke takes more effort than starting over.

The owner doesn't have to be an engineer. They need to understand the workflow, have access to the logs, and know the difference between a problem they can fix and one to escalate. Without an owner, the agent drifts.


One thing to do this week

For any agent workflow you're considering, write down what happens when the trigger fires with bad data. No company name in the form. A duplicate in the CRM. An API call that fails and returns nothing.

If you can't answer what the agent does in those cases, you're not ready to build yet.


Want me to look at yours?

Bring one workflow to a 30-minute call with me. I'll tell you exactly where an agent would fit in your business, what it would take to build, and whether it's even worth it for you. No pitch - just an honest read from someone who does this for a living.

Book a mapping call

Justin Henriksen

Justin Henriksen

Founder & CEO, GetLatest AI

Justin is the founder of GetLatest AI. 25 years building and leading technology, from Principal SWE to CEO. He writes about AI agent architecture, production systems, and what actually works.

Want your AI team built and run for you?

Book a free call30 min · no commitment