In February we put an AI assistant on WhatsApp in front of real customers of real businesses. It answers questions, checks what is available, books and cancels sessions, takes payment, and hands over to a person when it should.
Eight months later it has been rebuilt twice. This is what each version got right, where each one ran out of road, and what I would do differently if I started again on Monday.
The problem we were solving
Bizzly runs the front and back office for service businesses: kids activity clubs, cleaners, tutors, dog walkers, salons. The front office is their public website and the booking and checkout their customers use. The back office is the calendar, the customer records, the plans and packs, and payments through their own Stripe account. Same system, both sides.
One assumption is worth correcting early, because it shaped what we built. These are owner-run businesses, and owner-run does not mean small. One of our kids activity clients runs nine locations and takes five figures a month. Real scale, real payroll. What they do not have is a support department.
That is the actual problem. The owner is the support desk. Parents message at nine in the evening asking whether there is space on Saturday, whether a missed session can be moved, what they still owe. The questions repeat, the answers are already in the system, and the person answering them spent the day coaching. Nine locations produce nine locations worth of the same five questions.
So the assistant exists to answer a question the customer could have answered themselves if they had logged in, at the moment they actually asked, in the app they already use.
The risk that shapes everything
Ordinary software is wrong in ways you can reproduce. You find the input that breaks it, you fix it, you add a test, and that particular wrongness is gone for good. Fixing bugs makes the system more correct, permanently, one defect at a time.
An assistant does not behave like that, and the reason is not that it interprets what people mean. It is that the interpretation is not stable. The same sentence can be read two ways on two occasions. Two customers phrasing a request slightly differently can land in different places. A fault you saw yesterday may not reproduce today, which means you cannot fix it and prove it fixed in the way the rest of your career has trained you to.
Nor does it converge. Correct one misreading and another appears somewhere else, because the space of things people can type has no edge. You are not working through a finite list of defects. You are shifting the odds on an infinite surface, and you never get to say it is done.
Sit with that for a moment, because it is uncomfortable and it is the whole problem. Customers type however they like, so understanding has to be flexible. But a booking either exists or it does not, and a payment is either taken or it is not, so actions have to be exact. You have an unreliable component sitting in front of a system where exactness is the entire point.
The way out is not to make the interpretation reliable, because you cannot. It is to limit what an interpretation is allowed to cause. Every technique in the rest of this article is a version of that one sentence.
So treat the assistant as a channel, not a product
The most effective way to limit what a misreading can cause is to give the assistant nowhere of its own to act. Ours is a third way into a system that already existed. The website takes bookings. The staff screen takes bookings. The assistant takes bookings. All three obey one set of rules about who may book what, when, and at what price.
The mechanism matters as much as the principle. Those rules are not merely written down in one place, they are callable. Availability, entitlement, pricing and purchase rules sit behind an interface, and each way in is a client of it. That is what lets the assistant be a channel rather than a rewrite: it asks the same questions the website asks, in the same way, and gets the same answers.
The payoff is that "correct" was already defined. A child either has a session credit or does not, and that was true long before any AI was involved. We never had to decide what the right answer was, only how to reach it. More importantly, a misreading is contained: the worst the assistant can do is ask the same code the website asks, and get told no by the same rules.
The alternative is the default, and it is a trap. Give an assistant its own understanding of your business, its own notion of what a plan includes and when a booking is allowed, and you have written a second implementation of your company, on top of a component whose behaviour varies. It will be roughly right and it will drift. Every rule you forget to copy across is a defect you hear about from a customer rather than a test.
So the advice before any of the AI-specific lessons below, and it comes as a question you can answer this afternoon: can something other than your own web pages ask your system whether this customer may book that session, and get a trustworthy yes or no?
If the answer is yes, adding an assistant is a tractable project. If your rules live inside page controllers, form handlers and a few stored procedures, the assistant will not fix that. It will expose it. You will end up extracting your business logic first and building the assistant second, except you will be doing it under time pressure with a customer conversation as the test case.
The bar this sets is high, and deliberately so. Not "is that a good answer" but "is that exactly what the website would have done". Everything that follows is what it took to meet it.
Version one: one assistant, one big instruction
Measured against that idea, version one placed no limit on what an interpretation could cause. The AI read the message, decided what that meant, and acted on its own conclusion in a single movement. Nothing sat in the gap, because there was no gap.
What it was made of
- One loop. Take the message, ask the AI what to do, run whatever it asked for, send the reply.
- One long instruction telling the AI its role, its tone, what it could do, and the rules of the business.
- A set of actions it could call: look up a booking, check an account, take a payment.
- The conversation history, replayed back to the AI on every message.
What worked
More than I expected, and that is exactly the trap. It coped with the enormous variety of ways people phrase things from day one. Adding a new ability took an afternoon. It demos extremely well, which is how projects like this get approved.
Where it ran out of road
You can tell an AI a rule. You cannot prove it followed one. That gap does not matter for tone of voice. It matters enormously for money. An instruction and a guarantee are different things, and a design built mainly from instructions has no way to demonstrate the difference to anyone who asks for it: an auditor, an insurer, or the owner whose money is moving.
Nothing could be tested in isolation. Every decision happened in the same place at the same moment, so there was no individual behaviour to write a test against. We could judge whether conversations felt good. We could not prove any particular thing was true.
When something went wrong, you could not say what had gone wrong. You got a conversation and a shrug. There were no separate decisions to inspect, so every investigation was guesswork.
What that forced
The order of events and the question of who is allowed to do what had to become real parts of the system rather than sentences in an instruction. Put in the terms of the last section: we needed somewhere to stand between an interpretation and its consequences, and in this design there was nowhere to put it.
Version two: a map of the conversation
We moved to a graph, using LangGraph. Instead of one component deciding everything, a conversation now travels through named steps, and the route between them is written in code.
The effect was immediate and physical. The main file shrank by hundreds of lines, and the long instruction lost about half its length, because most of it had been describing what should happen next. The map absorbed the instructions.
This was the first genuine limit on a misreading, and it is worth being precise about what kind. It constrained where a conversation could go. It did not yet constrain what it could do once it got there.
What it was made of
- A classifier at the door, working out what the message is about and returning it in a fixed format rather than prose.
- Routing rules, ordinary code deciding which step handles this message.
- Steps, each responsible for one job: booking, cancelling, account questions, help.
- A structured record of the conversation, so what someone already chose is stored as a fact rather than re-read from the chat each time.
- One list of available actions, defined in a single place.
What worked
You can see the path a conversation took. That alone changed how quickly we could fix things.
Interruptions stopped being a problem. Someone halfway through booking who asks what it costs, then comes back, is completely normal behaviour. Storing the conversation properly handles it.
We could finally write tests. Most of ours check the route a conversation takes and what was stored, rather than the wording of a reply. That only becomes possible once routes and stored facts exist as real things.
Where it ran out of road
A map controls the order of things, not what they are allowed to do. Each step carried its own checks, which is perfectly manageable with four steps and increasingly unwise as you approach twenty. Correctness was spread across every step rather than established in one shared place, so confirming how the system behaves meant reading all of it rather than one file. That does not scale, and it is exactly the kind of property that quietly decays as a codebase and a team grow.
Checking identity at the start protected the system and lost us conversations.Anyone unrecognised asking about a session was sent a code before they learned anything. It is defensible and it is a wall. People arrive with a question, not their credentials, and a good number of them left rather than prove who they were to find out whether Saturday was even free.
Anything the assistant knew instead of looked up went stale. This is the general version of a specific lesson. The moment a business changes a price, a policy or a session length, anything the AI was told at the start is now wrong, and it will say it with complete confidence.
The field is watching the wrong door. Almost all public discussion of AI safety concerns what people send in. The harder problem in our experience is what goes out: fluent, plausible replies that are subtly wrong about that business's own rules. No amount of filtering the incoming message catches one of those, because the message was perfectly reasonable. The check has to sit on the reply.
What that forced
Four things, and together they are the current version: something standing between a step and a real action, identity checked at the moment it matters rather than at the door, facts looked up rather than remembered, and replies checked before they go out.
All four are the same move. Version two bounded which path a misreading could take. Version three had to bound what the path was able to do at the end of it.
Version three: the one running now
The shape has not changed since. What changed is that every constraint named earlier got a real place to live. This is the version in which a misreading became survivable rather than merely unlikely, which is a different and much more comfortable engineering position.
An AI asking to book something has not booked anything
This is the single most useful idea to take away, and the phrase "tool calling" hides it.
When the AI decides to book a session, nothing has been booked. It has made a request. Eight questions sit between that request and a real booking, and the AI cannot answer any of them:
- Who is this? Verified, not claimed.
- Which business? Decided by the system, never taken from the message.
- Are they allowed to do this at all?
- Is there actually space? Right now, not when the list was shown.
- Are they entitled to this session? Does their plan or pack cover it.
- Does the business allow this purchase? For this person, in this window.
- Have we already done this? People and phone networks both repeat themselves, and nobody should be charged twice.
- Does this step make sense right now? Given where the conversation has got to.
Every one of those checks already existed, because the website and the staff screen had needed them for years. The work was not inventing them. It was making sure the assistant went through them rather than around them.
Two requests in one sentence
"Move Amelia to Saturday and cancel Tom next week" is one message and two actions. This is the case where an assistant is genuinely more useful than a form, and also the one where acting on a half-understood request does the most damage.
So the AI does not carry it out. It writes a plan: an ordered list of steps in a fixed format, each naming what to do and with what. That plan is then checked as a whole before any of it runs. A plan whose second step needs something the first step never obtained is rejected up front, rather than discovered halfway through with one of the two actions already done. Once it passes, the steps run one at a time, and each goes through exactly the same questions as any other action.
The part worth copying is the split. The planner cannot do anything. It has no ability to book, cancel or charge. It writes a list, and something else decides whether to run it. The component with the most freedom has the least authority, which is the same idea as everything else here, applied to the riskiest kind of request.
Check identity at the moment it matters
This replaced the wall at the front door, and it is the change I would make first in any similar system.
The conversation now runs freely until something irreversible is about to happen. Someone can ask what is on, see real availability, and pick a session. The moment they pick, their choice is set aside, we check who they are, and the booking continues afterwards with that same choice.
The principle generalises: being strict is expensive for the person on the other end, so spend it where the consequence is, not where the conversation starts.
Look things up. Do not remember them.
This is where most of the risk of the assistant making things up actually lived, and there are two separate problems stacked on top of each other.
The first is working out which business the conversation is even about.The assistant runs on our own WhatsApp number and on each business's number, and which one a message arrived on is decided before anything else happens. A message to a club's number is a parent talking to that club. A message to our number is a business owner, or someone thinking of becoming one, talking to us about the software.
The same sentence means different things on the two lines, and the answer has to come from a different place entirely.
| The same question | On a business's own number | On our number |
|---|---|---|
| Where do you run sessions? | Search that business's own website | Search our own website |
| How do I change my plan? | Read that customer's account with the business | Our help centre, written for owners |
| What does it cost? | That business's price list | Our pricing |
| Is there space on Saturday? | That business's calendar | Nothing here to answer from |
Miss that and you produce answers that are fluent, sincere and about the wrong company. The mechanism is worth understanding because nothing about it looks broken: a search that runs without knowing who is asking returns the closest matching text in the whole library, rather than the closest text that person should be reading. Those are different answers, and only one of them is right.
That also means the help centre cannot be searched as one undifferentiated pile. An article written for owners is the wrong answer for a parent even when it is the closest match, so who is asking has to narrow the search before it runs. When we do not yet know who someone is, we assume the safer of the two: a customer of the business on a club's line, and someone learning about the software on ours.
The second problem is that questions come in two kinds, and they need opposite treatment. Some are questions about words. A business wrote a page about where it operates, and we wrote help articles about the software. Searching those by meaning works well, because an approximate match is genuinely useful.
Others are questions about facts. A price, a free space, a remaining session balance. There is one correct answer at this exact moment, and it gets read straight out of the system that owns it.
The cost of this is unglamorous, and it is the real reason this took eight months rather than one. Every category of fact the assistant may discuss needs a proper connection to the system that owns it, and every search needs to know whose library it is reading.
One route to payment
The assistant does not have its own way of taking money. It drives the same checkout the staff screen uses, through the same checks. If a purchase is not allowed for a customer, it is not allowed in chat either, and nobody has to remember to write the rule twice.
Check the reply before it goes out
Certain replies are checked against the business's rules before sending, and anything that breaks one is replaced with a safe answer rather than patched up.
The failure this catches is worth naming precisely, because it is not the one people expect. It is a reply that is well written, internally consistent, and about the wrong thing. Filtering the incoming message never sees it coming, because the customer's question was perfectly ordinary. The only place to catch it is on the way out, by asking whether this answer is one this business would stand behind.
You cannot test everything, so watch everything
Normal testing assumes you can list the possible inputs. Here the input is anything a person might type. Three things replaced that, and together they are what actually drove improvement.
Every conversation is recorded as data, not just logged: which steps ran, how long it took, what it cost, what went wrong. The useful questions are all totals. Which kind of request is slowest, which business generates the most failures, whether last week's change made anything worse.
The thing you debug is the conversation, not the message. Ordinary software monitoring looks at one request at a time, which is useless here, because the problem is almost never in one message. It is that message four established something that message seven contradicted. So we read whole conversations, with each turn showing what the AI thought it was about and what it did. Nearly every fix in this article started with someone reading a conversation and asking why it went that way.
A simulator runs scripted conversations through the real system with saving turned off. Not a copy of the system, the real one, from a real account. It answers the assistant as a customer would, and the results read exactly like production conversations. This is not proper test coverage and I would not pretend it is. What it buys is the confidence to change things. Before it existed we changed less than we should have, because every change was a gamble.
The same checks run twice. The rules that stop a bad reply going out also run over the past week of real conversations on a schedule, and shout if anything slipped through. Something caught that way becomes a scripted conversation in the simulator, and that stops it coming back.
Read those five together and they are one idea applied five times. The assistant can still misread someone. What it can no longer do is turn a misreading into a booking for the wrong person, a charge nobody agreed to, a confident quote of a price that is not real, or an answer about the wrong company. The interpretation stayed unreliable. The consequences stopped being open-ended.
What is still hard
Working out what a message is about is a single point of failure. Get that wrong and the customer goes down the wrong path, and no amount of correctness further down rescues it.
Every protection you add catches innocent people. We limit how much an unverified number can consume, which is sensible for something that costs money per message. Any limit like that has a false positive rate. The lesson is not "do not add limits". It is to design each one as though the person it catches is your best customer halfway through buying something, because sooner or later it will be, and to go looking for who else it catches rather than waiting to be told.
The AI's instructions are still the most edited part of the system.After eight months of moving decisions out of them, they still change more often than anything else. We now treat that as a signal. If we are adding a sentence to stop a behaviour, that usually means a decision belongs somewhere more solid.
What I would do differently
Everything above is a principle we arrived at by building. Starting again on Monday, three things would be different. None are technically difficult. Each is a way of settling up front what an interpretation is allowed to cause, rather than working it out as you go.
Write down what the AI is never allowed to decide, first
Before any code. Ours ended up as: it never decides whether someone is verified, what a price is, whether a purchase is allowed, whether there is space, or whether a reply is safe to send.
Write it before the first line of code, not after the first incident. It takes an afternoon, and it is the most useful document about the system: the only one that answers what this thing can do to a customer without anyone having to read the codebase.
Rank actions by how bad it is to get them wrong
Not everything needs the same protection, and treating it as one setting is why we moved the same line repeatedly. Answering a question and taking eighty pounds are not the same kind of act.
| The action | If it goes wrong | What it has to pass first |
|---|---|---|
| Answer a question | One more message to put it right | The reply is checked before it goes out |
| Send a message out | Money per send, and goodwill | The above, plus a limit on how much one number can trigger |
| Book or cancel a session | Takes someone else's place, awkward to undo | The above, plus proof of who they are, a live check the place exists, and whether their plan covers it |
| Take payment | Real money and a real dispute | The above, plus what the business allows to be sold, protection against charging twice, and the same checkout the staff screen uses |
Read the last column downwards: each row keeps everything above it and adds one more requirement. That is the whole idea. Sort every action into a band, decide once what each band requires, and a new feature inherits its protection from the band it lands in rather than having the argument again from scratch.
We arrived at this by accident, one capability at a time, which is why the same line kept moving. Written down on day one it is an afternoon of work and it settles dozens of later decisions.
Build the tools that let you see and rerun conversations, first
The conversation viewer, the simulator and the recorded history are not things you add to a working system. They are what make a system improvable at all. They belong before the second feature, not after the tenth.
Where freedom helps, and where it costs
If you take one thing from this article, take this table. "How much freedom should the AI have" is the wrong question, because the answer is different for every decision.
| The decision | Who makes it | Why |
|---|---|---|
| What the customer meant | The AI | People phrase things a thousand ways, and a wrong guess costs one follow-up question |
| How the reply is worded | The AI | This is the entire reason customers put up with the channel |
| Getting back on track after confusion | The AI | Rigid systems dead-end here and people give up |
| Which facts are worth looking up | The AI chooses, the system fetches | Choosing is a language problem, the answer is not |
| What a price or policy is | The system | Anything the AI remembers is out of date the moment a business changes it |
| Whether this person is who they say | The system | A phone number is a claim, not proof |
| Whether an action is allowed | The system | Permissions are not a matter of interpretation |
| What gets charged, and to whom | The system | One route to payment, with the checks staff already use |
| Whether the reply is safe to send | The system | A confident, well-written, wrong answer is the normal failure |
There is a clean split here. Freedom helps when the cost of being wrong is one more message. It hurts when the cost of being wrong is a charge, a booking, or telling someone something they should not know. The second kind does not improve with a better AI model, because that class of failure is not about intelligence. It is about which component holds the decision. A judgement call belongs to the AI. A rule belongs in code, where it can be stated once and shown to hold.
What "agentic" turned out to mean
First, the plain answer: it worked. A parent messages at nine in the evening, in their own words, about a specific child in a specific class, and gets a correct answer. They can move a session, buy a pack, cancel one, ask what they are owed. The number of ways people phrase those things is endless and no menu was ever going to cover it. It handles operations a traditional chatbot could not attempt, and it does so against live data with real money involved.
So the goal was met. What I want to question is the word we have all agreed to use for it.
When people talk about agentic AI, the picture in the room is of the AI doing the heavy lifting: working out what is needed, deciding whether it is allowed, and carrying it out. Autonomy is the headline. That is not what made this work, and I suspect it is not what made anyone else's work either.
What made it work was a business whose processes were already pinned down and already reachable by something other than a web page. Availability, entitlement, what may be sold to whom, what a cancellation means: all of it defined, enforced in one place, correct before any AI existed, and exposed through an API that anything could call.
That last clause is not a technicality. A rule that exists only inside a screen is not a rule an assistant can use, and the gap between "we know our process" and "our process is callable" is where most of the real work sits. The AI supplies a natural language way in to processes that were already well understood and already addressable. It does the part that genuinely needs flexibility, which is understanding people, and it is kept away from the part that needs to be right.
That is not a disappointment, and I do not want it read as one. It is a precondition, and knowing it up front would have saved us months of looking for the sophistication in the wrong layer.
What I genuinely cannot tell you is where the line sits. How much decision-making can you hand to the AI before it becomes a mess? We have been conservative, and every time we extended its reach we ended up pulling some of it back. But I cannot tell you whether that is the real ceiling or simply where our nerve ran out.
My instinct is that the limit tracks reversibility rather than difficulty. Decisions you can undo cheaply are safe to hand over, and the AI is often better at them than a rigid rule would be. Decisions that move money, allocate a scarce place, or say something to a customer that cannot be unsaid want a rule with a name and a test. But "instinct" is doing a lot of work in that sentence, and anyone who tells you they have measured this precisely is guessing with more confidence than I am.
Seven lessons
Eight months, three architectures, and these are what I would hand someone on day one.
Let the AI interpret and decide. Never let it execute. It is better than any rule I could write at two things: understanding what somebody meant, and working out what ought to happen next. It is worse than any rule at a third: doing the same thing the same way twice. That third one is the property a business process cannot survive without. Hand over the judgement. Keep the doing.
You are funding a platform project with an AI deadline attached. The AI was the cheap part. What actually decided when we could go live was making the business rules consistent, and reachable by something other than our own web pages. That is the work sponsors under-scope, every time, because it does not demo.
It is also the only part of this with a growth curve rather than a maintenance curve. The same entitlement checks, purchase rules and checkout the assistant uses are the ones the website and the staff screen use, so every future channel inherits them for nothing. Which gives you a planning question you can answer this week: whatever is not yet behind that layer is the hidden cost of your next AI project.
Budget for one structural rewrite, and stop treating it as failure.Three architectures in eight months, and every change was forced by a problem we had not anticipated rather than chosen from a roadmap. You cannot specify these risks up front, because they live in what real customers type. So plan the rewrite as a cost of the programme. A team that has not rewritten anything has probably not learned anything yet.
Write down what the AI may never decide, before you write any code.Ours: who someone is, what a price is, whether a purchase is allowed, whether there is space, and whether a reply is safe to send. It is an afternoon of work, and it is the only document that answers "what can this thing do to a customer" without reading the codebase. Write it first and it shapes the architecture. Arrive at it later and you are retrofitting it into decisions already taken.
Put your strictness where the consequence is, then check who it catches.Asking someone to prove who they are before they have learned anything protects the system and loses you the customer. Move it to the moment something becomes irreversible, and rank every action by how easily it can be undone.
Then the part most teams skip. A control tuned only against the abuse case will catch good-faith customers, and it will do it at the worst possible moment. Every new guard needs its false positives reviewed before it ships, not just its false negatives.
Assume the damage comes out, not in. Almost all published AI safety advice concerns what people send you. The expensive failures are fluent, confident replies that are wrong about your business, and nothing upstream catches them because the customer asked a perfectly reasonable question.
Worse, the context that makes an answer correct does not stay proven. Which business this conversation is about, and who is asking, must be re-established by every capability you add. A search or a pricing path built in month six does not inherit it from one built in month two.
Verification is a permanent second engineering track, not a phase. You cannot test your way to confidence, because the same input does not reliably reproduce the same failure. Conversation capture, replay and checks that run over real traffic are not a one-off investment you pay down. They have to grow with the system. Fund them once and you will slowly lose the ability to say whether the thing still works.
One last observation, for anyone hoping to wait this problem out. Over these eight months the models got materially better. The authority we gave the assistant did not expand to match. Every extension of its reach ended up narrower than it started, never wider. Model progress did not move our ceiling, because the constraint was never the model's intelligence.
Which sharpens what I said earlier about not knowing where the line sits. For anything the assistant does, the line is not mysterious at all: it is exactly the set of actions that already have a callable, tested, tenant-scoped rule behind them, and it moves as fast as you shorten that list. That is a punch list, not a philosophy question, and it is fundable.
The genuinely open question is narrower than I first made it sound. It is about the decisions where no rule could exist, because the whole value is judgement: who should be offered the place that just came free. There is nothing to write down there, and no amount of platform work settles it. That is the part we will have to find out by trying, somewhere a wrong answer costs nothing but a second offer.