Agentic AI in production: four architectures, eight lessons

In February we put an AI assistant on WhatsApp in front of real customers of real businesses. It answers questions, checks what is available, books and cancels sessions, takes payment, and hands over to a person when it should.

Eight months later it has been rebuilt three times, the last time in September, and in October the layer beneath it was rebuilt too. This is what each version got right, where each one ran out of road, and what I would do differently if I started again on Monday.

The eight lessons, for anyone who reads the end first:

1. Make the process callable before you write a prompt.
2. Let the AI interpret and decide. Never let it execute.
3. Write the policy down, and make it callable.
4. Budget for one structural rewrite, and stop treating it as failure.
5. Put your strictness where the consequence is, then check who it catches.
6. Assume the damage comes out, not in.
7. Verification is a permanent second engineering track, not a phase.
8. Once the rails are real, give the loop its freedom back.

The problem we were solving

Bizzly runs the front and back office for service businesses: kids activity clubs, cleaners, tutors, dog walkers, salons. The front office is their public website and the booking and checkout their customers use. The back office is the calendar, the customer records, the plans and packs, and payments through their own Stripe account. Same system, both sides.

One assumption is worth correcting early, because it shaped what we built. These are owner-run businesses, and owner-run does not mean small. One of our kids activity clients runs nine locations and takes five figures a month. Real scale, real payroll. What they do not have is a support department.

That is the actual problem. The owner is the support desk. Parents message at nine in the evening asking whether there is space on Saturday, whether a missed session can be moved, what they still owe. The questions repeat, the answers are already in the system, and the person answering them spent the day coaching. Nine locations produce nine locations worth of the same five questions.

So the assistant exists to answer a question the customer could have answered themselves if they had logged in, at the moment they actually asked, in the app they already use.

The risk that shapes everything

Ordinary software is wrong in ways you can reproduce. You find the input that breaks it, you fix it, you add a test, and that particular wrongness is gone for good. Fixing bugs makes the system more correct, permanently, one defect at a time.

An assistant does not behave like that, and the reason is not that it interprets what people mean. It is that the interpretation is not stable. The same sentence can be read two ways on two occasions. Two customers phrasing a request slightly differently can land in different places. A fault you saw yesterday may not reproduce today, which means you cannot fix it and prove it fixed in the way the rest of your career has trained you to.

Nor does it converge. Correct one misreading and another appears somewhere else, because the space of things people can type has no edge. You are not working through a finite list of defects. You are shifting the odds on an infinite surface, and you never get to say it is done.

Sit with that for a moment, because it is uncomfortable and it is the whole problem. Customers type however they like, so understanding has to be flexible. But a booking either exists or it does not, and a payment is either taken or it is not, so actions have to be exact. You have an unreliable component sitting in front of a system where exactness is the entire point.

The way out is not to make the interpretation reliable, because you cannot. It is to limit what an interpretation is allowed to cause. Every technique in the rest of this article is a version of that one sentence.

So treat the assistant as a channel, not a product

The most effective way to limit what a misreading can cause is to give the assistant nowhere of its own to act. Ours is a third way into a system that already existed. The website takes bookings. The staff screen takes bookings. The assistant takes bookings. All three obey one set of rules about who may book what, when, and at what price.

The mechanism matters as much as the principle. Those rules are not merely written down in one place, they are callable. Availability, entitlement, pricing and purchase rules sit behind an interface, and each way in is a client of it. That is what lets the assistant be a channel rather than a rewrite: it asks the same questions the website asks, in the same way, and gets the same answers.

The payoff is that "correct" was already defined. A child either has a session credit or does not, and that was true long before any AI was involved. We never had to decide what the right answer was, only how to reach it. More importantly, a misreading is contained: the worst the assistant can do is ask the same code the website asks, and get told no by the same rules.

The alternative is the default, and it is a trap. Give an assistant its own understanding of your business, its own notion of what a plan includes and when a booking is allowed, and you have written a second implementation of your company, on top of a component whose behaviour varies. It will be roughly right and it will drift. Every rule you forget to copy across is a defect you hear about from a customer rather than a test.

So the advice before any of the AI-specific lessons below, and it comes as a question you can answer this afternoon: can something other than your own web pages ask your system whether this customer may book that session, and get a trustworthy yes or no?

If the answer is yes, adding an assistant is a tractable project. If your rules live inside page controllers, form handlers and a few stored procedures, the assistant will not fix that. It will expose it. You will end up extracting your business logic first and building the assistant second, except you will be doing it under time pressure with a customer conversation as the test case.

The bar this sets is high, and deliberately so. Not "is that a good answer" but "is that exactly what the website would have done". Everything that follows is what it took to meet it.

Version one: one assistant, one big instruction

Measured against that idea, version one placed no limit on what an interpretation could cause. The AI read the message, decided what that meant, and acted on its own conclusion in a single movement. Nothing sat in the gap, because there was no gap.

What it was made of

• One loop. Take the message, ask the AI what to do, run whatever it asked for, send the reply.
• One long instruction telling the AI its role, its tone, what it could do, and the rules of the business.
• A set of actions it could call: look up a booking, check an account, take a payment.
• The conversation history, replayed back to the AI on every message.

What worked

More than I expected, and that is exactly the trap. It coped with the enormous variety of ways people phrase things from day one. Adding a new ability took an afternoon. It demos extremely well, which is how projects like this get approved.

Where it ran out of road

You can tell an AI a rule. You cannot prove it followed one. That gap does not matter for tone of voice. It matters enormously for money. An instruction and a guarantee are different things, and a design built mainly from instructions has no way to demonstrate the difference to anyone who asks for it: an auditor, an insurer, or the owner whose money is moving.

Nothing could be tested in isolation. Every decision happened in the same place at the same moment, so there was no individual behaviour to write a test against. We could judge whether conversations felt good. We could not prove any particular thing was true.

When something went wrong, you could not say what had gone wrong. You got a conversation and a shrug. There were no separate decisions to inspect, so every investigation was guesswork.

What that forced

The order of events and the question of who is allowed to do what had to become real parts of the system rather than sentences in an instruction. Put in the terms of the last section: we needed somewhere to stand between an interpretation and its consequences, and in this design there was nowhere to put it.

Version two: a map of the conversation, on LangGraph

We moved to a graph, using LangGraph. Instead of one component deciding everything, a conversation now travels through named steps, and the route between them is written in code.

The effect was immediate and physical. The main file shrank by hundreds of lines, and the long instruction lost about half its length, because most of it had been describing what should happen next. The map absorbed the instructions.

This was the first genuine limit on a misreading, and it is worth being precise about what kind. It constrained where a conversation could go. It did not yet constrain what it could do once it got there.

What it was made of

• A classifier at the door, working out what the message is about and returning it in a fixed format rather than prose.
• Routing rules, ordinary code deciding which step handles this message.
• Steps, each responsible for one job: booking, cancelling, account questions, help. In the diagram's terms these are worker agents, though at this point they were mostly code.
• A structured record of the conversation, so what someone already chose is stored as a fact rather than re-read from the chat each time.
• One list of available actions, defined in a single place.

What worked

You can see the path a conversation took. That alone changed how quickly we could fix things.

Interruptions stopped being a problem. Someone halfway through booking who asks what it costs, then comes back, is completely normal behaviour. Storing the conversation properly handles it.

We could finally write tests. Most of ours check the route a conversation takes and what was stored, rather than the wording of a reply. That only becomes possible once routes and stored facts exist as real things.

Where it ran out of road

A map controls the order of things, not what they are allowed to do. Each step carried its own checks, which is perfectly manageable with four steps and increasingly unwise as you approach twenty. Correctness was spread across every step rather than established in one shared place, so confirming how the system behaves meant reading all of it rather than one file. That does not scale, and it is exactly the kind of property that quietly decays as a codebase and a team grow.

Checking identity at the start protected the system and lost us conversations.Anyone unrecognised asking about a session was sent a code before they learned anything. It is defensible and it is a wall. People arrive with a question, not their credentials, and a good number of them left rather than prove who they were to find out whether Saturday was even free.

Anything the assistant knew instead of looked up went stale. This is the general version of a specific lesson. The moment a business changes a price, a policy or a session length, anything the AI was told at the start is now wrong, and it will say it with complete confidence.

The field is watching the wrong door. Almost all public discussion of AI safety concerns what people send in. The harder problem in our experience is what goes out: fluent, plausible replies that are subtly wrong about that business's own rules. No amount of filtering the incoming message catches one of those, because the message was perfectly reasonable. The check has to sit on the reply.

What that forced

Four things, and together they made version three: something standing between a step and a real action, identity checked at the moment it matters rather than at the door, facts looked up rather than remembered, and replies checked before they go out.

All four are the same move. Version two bounded which path a misreading could take. Version three had to bound what the path was able to do at the end of it.

Version three: the map, with rails

This is the version that ran until this month. The map stayed. What changed is that every constraint named earlier got a real place to live, and each of those places outlived the map itself, which is why they get their own headings below. This is the version in which a misreading became survivable rather than merely unlikely, which is a different and much more comfortable engineering position.

An AI asking to book something has not booked anything

This is the single most useful idea to take away, and the phrase "tool calling" hides it.

When the AI decides to book a session, nothing has been booked. It has made a request. Eight questions sit between that request and a real booking, and the AI cannot answer any of them:

• Who is this? Verified, not claimed.
• Which business? Decided by the system, never taken from the message.
• Are they allowed to do this at all?
• Is there actually space? Right now, not when the list was shown.
• Are they entitled to this session? Does their plan or pack cover it.
• Does the business allow this purchase? For this person, in this window.
• Have we already done this? People and phone networks both repeat themselves, and nobody should be charged twice.
• Does this step make sense right now? Given where the conversation has got to.

Every one of those checks already existed, because the website and the staff screen had needed them for years. The work was not inventing them. It was making sure the assistant went through them rather than around them.

Two requests in one sentence

"Move Amelia to Saturday and cancel Tom next week" is one message and two actions. This is the case where an assistant is genuinely more useful than a form, and also the one where acting on a half-understood request does the most damage.

In version three the AI did not carry it out. It wrote a plan: an ordered list of steps in a fixed format, each naming what to do and with what. That plan was then checked as a whole before any of it ran, and once it passed the steps ran one at a time, each through exactly the same questions as any other action.

Version four, below, dropped the plan and kept the split. The AI now works through the two actions itself, one at a time within the same turn, and each still goes through the same questions. What makes that safe is that neither action can happen on the AI's say-so: each is quoted back to the customer first, and only their yes to that exact quote can cause it. The part worth copying is the same either way. The component with the most freedom has the least authority, which is the idea behind everything else here, applied to the riskiest kind of request.

Check identity at the moment it matters

This replaced the wall at the front door, and it is the change I would make first in any similar system.

The conversation now runs freely until something irreversible is about to happen. Someone can ask what is on, see real availability, and pick a session. The moment they pick, their choice is set aside, we check who they are, and the booking continues afterwards with that same choice.

The principle generalises: being strict is expensive for the person on the other end, so spend it where the consequence is, not where the conversation starts.

Look things up. Do not remember them.

This is where most of the risk of the assistant making things up actually lived, and there are two separate problems stacked on top of each other.

The first is working out which business the conversation is even about.There is one WhatsApp number for everyone, so the number tells us nothing. A conversation is tied to a business by the message that opens it: the chat button on a club's website starts the conversation with a line naming that site, and from then on every message in it is a parent talking to that club. A conversation nothing ties to a business is an owner, or someone thinking of becoming one, talking to us about the software. That is decided before anything else happens, and never from what the message says it wants.

So there are two contexts, and each has its own sources of truth: a tenant on Bizzly, one business talking to its own customers, and the Bizzly platform, talking to the people who run those businesses. The same sentence means different things in each, and the answer comes from a different place entirely.

Miss that and you produce answers that are fluent, sincere and about the wrong company. The mechanism is worth understanding because nothing about it looks broken: a search that runs without knowing who is asking returns the closest matching text in the whole library, rather than the closest text that person should be reading. Those are different answers, and only one of them is right.

That also means the help centre cannot be searched as one undifferentiated pile. An article written for owners is the wrong answer for a parent even when it is the closest match, so who is asking has to narrow the search before it runs. When we do not yet know who someone is, we assume the safer of the two: a customer of the business when a business is tied to the conversation, and someone learning about the software when nothing is.

The second problem is that questions come in two kinds, and they need opposite treatment. Some are questions about words. A business wrote a page about where it operates, and we wrote help articles about the software. Searching those by meaning works well, because an approximate match is genuinely useful.

Others are questions about facts. A price, a free space, a remaining session balance. There is one correct answer at this exact moment, and it gets read straight out of the system that owns it.

The cost of this is unglamorous, and it is the real reason this took eight months rather than one. Every category of fact the assistant may discuss needs a proper connection to the system that owns it, and every search needs to know whose library it is reading.

How the searching works, and what it is not for

The searching half is what the industry calls retrieval-augmented generation, and it is worth being concrete about, because the phrase hides both how ordinary it is and where it goes wrong.

What is indexed. Each business's own website pages, its service, plan and price cards and its venue timetables, and separately our help centre. Every page is split into passages, each passage is turned into a numerical description of its meaning, and those sit in the database next to everything else. A scheduled job re-reads the live pages and refreshes them, so a business that rewrites a page is searchable the same day without anyone telling us.

How a search runs. The AI decides that a question is one a page might answer, and phrases the search. The phrase is described the same way the passages were, the closest passages come back, with a page whose title matches the question counted in its favour, and each passage is resolved back to the whole page it came from, so the AI reads the page rather than a fragment of it. Every search is scoped to one business before it runs, and a help centre search is scoped to who is asking, so an owner's article is never the answer to a parent's question however close the wording.

What it is not for. Anything with one true value at this moment. A price, a free place, a remaining balance, whether a plan covers a session: those are read straight from the system that owns them, never retrieved. And the business's configured services, products and rules go into every conversation up front, so the assistant never has to go searching for its own price list.

The trap retrieval sets is exactly the one the rule above guards against. A search returns the closest passage, and the closest passage about pricing is very often another product's. Retrieval is superb at "where do you run sessions" and dangerous at "what does it cost", and the whole design of this layer is keeping those two kinds of question apart.

One route to payment

The assistant does not have its own way of taking money. It drives the same checkout the staff screen uses, through the same checks. If a purchase is not allowed for a customer, it is not allowed in chat either, and nobody has to remember to write the rule twice.

Check the reply before it goes out

Certain replies are checked against the business's rules before sending, and anything that breaks one is replaced with a safe answer rather than patched up.

The failure this catches is worth naming precisely, because it is not the one people expect. It is a reply that is well written, internally consistent, and about the wrong thing. Filtering the incoming message never sees it coming, because the customer's question was perfectly ordinary. The only place to catch it is on the way out, by asking whether this answer is one this business would stand behind.

You cannot test everything, so watch everything

Normal testing assumes you can list the possible inputs. Here the input is anything a person might type. Three things replaced that, and together they are what actually drove improvement.

Every conversation is recorded as data, not just logged: what it looked up and did, how long it took, what it cost, what went wrong. The useful questions are all totals. Which kind of request is slowest, which business generates the most failures, whether last week's change made anything worse.

The thing you debug is the conversation, not the message. Ordinary software monitoring looks at one request at a time, which is useless here, because the problem is almost never in one message. It is that message four established something that message seven contradicted. So we read whole conversations, with each turn showing what the AI thought it was about and what it did. Nearly every fix in this article started with someone reading a conversation and asking why it went that way.

A simulator runs scripted conversations through the real system with saving turned off. Not a copy of the system, the real one, from a real account. It answers the assistant as a customer would, and the results read exactly like production conversations. This is not proper test coverage and I would not pretend it is. What it buys is the confidence to change things. Before it existed we changed less than we should have, because every change was a gamble.

The same checks run twice. The rules that stop a bad reply going out also run over the past week of real conversations on a schedule, and shout if anything slipped through. Something caught that way becomes a scripted conversation in the simulator, and that stops it coming back.

Read those five together and they are one idea applied five times. The assistant can still misread someone. What it can no longer do is turn a misreading into a booking for the wrong person, a charge nobody agreed to, a confident quote of a price that is not real, or an answer about the wrong company. The interpretation stayed unreliable. The consequences stopped being open-ended.

Where it ran out of road

Working out what a message is about was a single point of failure. Get that wrong and the customer went down the wrong path, and no amount of correctness further down rescued it. The AI understood people better than the component sorting their messages did, and the sorting was where most of our fixes went.

Every protection you add catches innocent people. We limit how much an unverified number can consume, which is sensible for something that costs money per message. Any limit like that has a false positive rate. The lesson is not "do not add limits". It is to design each one as though the person it catches is your best customer halfway through buying something, because sooner or later it will be, and to go looking for who else it catches rather than waiting to be told.

The map became the thing we edited most. A conversation that did not fit a route got a new route, and a customer who said something the current step did not expect got a sentence written in code to meet it. After eight months of that, the map held a great deal of language that the AI would have handled better, and every route had its own small copy of the same behaviour.

What that forced

A question we should have asked sooner. The rails were real by then: identity, permissions, facts, payment and the reply check all lived in code, at the point where a request becomes an action, and none of them depended on the map. So what was the map still for? Its one remaining job was deciding what the customer meant and what should happen next, and that is the job the AI does best.

Version four: one loop again, with the rails outside it

So this month we went back to the shape of version one. One assistant reads the whole conversation and decides each step itself: look something up, ask a question, or answer. The difference from version one is everything around it. In version one the loop held every decision. In version four it holds the two that belong to it, what the customer meant and what to say, and every other decision named in this article sits in code between the loop and the world.

That is also why version one's three failures do not come back with its shape. A rule the AI is told cannot be proven; a rule in code can, and the rules are in code. Nothing could be tested in isolation; now each check and each action is tested on its own, and the loop is tested on replayed conversations. Nothing said what had gone wrong; now every turn records what the AI looked up, what it asked for, and what it was refused.

What it is made of

• One assistant and the whole conversation. It sees what was said, and what it looked up on earlier turns, so a list it offered two messages ago is still in front of it when the customer replies with a number.
• The AI chooses what to ask the system. Availability, prices, an account, a booking: it decides which of those it needs and in what order. Code decides whether it is allowed the answer.
• The same rails as version three, in one place: identity checked when it matters, one route to payment, facts looked up rather than remembered, and the reply checked before it goes out.
• Nothing irreversible on the AI's say-so. A booking, a cancellation or a payment link is quoted to the customer first. Their yes is tied to that exact quote, so it can cause that and nothing else, and saying yes twice cannot cause it twice.
• The step-by-step parts are forms, not conversation. Booking and checkout run through the same guided sequence the website uses, one question at a time, with every option and every rule decided by that sequence. The AI fills the form in from what the customer already said.
• Messages that arrive mid-thought join the thought. Three messages in ten seconds get one reply that answers all three, and if a turn is cut off, the next message picks up with a record of what the cut-off turn already did.
• The AI writes every sentence. There is no reply written in code, only a handful of fixed lines for the moments no assistant can run at all.

What worked

The conversations that used to go wrong now go right, and we can show it, because we replay real conversations from the archive through each new version. A parent who had given the venue, the age and the day, and been asked for them again, gets the dates on the second message and is never asked twice. A burst of four messages typed in ten seconds gets one answer that covers all four.

There is about half as much of it. The map, the component that sorted messages, the routes and the state they carried between them all went, along with every reply that had been written in code. What is left is the assistant, the actions it can ask for, and the checks in front of them.

The freedom went to the right place. Version two took freedom away from the AI and gave it to a map. Version four gave it back, but only the part the AI is good at: reading people and choosing what to do next. Its authority over anything that costs money or takes a place is exactly what it was, which is none.

What is still hard

Freedom moves the work from routes to evidence. With a map, improving the assistant meant adding a route, and you could see that it was there. With a loop, it means giving the AI a better instruction or a better tool, and the only way to know either worked is to run the conversations that mattered through it again. So the replays are the test suite now. Every archived conversation worth keeping runs against every change, and that library has to keep growing, because a loop without evidence is guesswork with good manners.

Reading the whole conversation costs more per message than reading a route.Most of it is served from a cache, and the rounds that repeated a lookup are gone, but a long conversation is a long prompt, and a fact given forty messages ago can fall out of the window the assistant reads. So it keeps a small written memory of settled facts, where, who it is for and whatever that business's profile form asks about them, which service, a preferred day, that it writes itself the moment one is settled and reads on every turn. A handful of short notes, capped, and never contact or account details, so the note can never grow into a second copy of the account.

The AI's instructions are still the most edited part of the system.After eight months of moving decisions out of them, they still change more often than anything else. We now treat that as a signal. If we are adding a sentence to stop a behaviour, that usually means a decision belongs somewhere more solid.

Under the loop: the process became callable

The loop has not changed since September. What changed in October is what it calls. Lesson one says make the process callable, and for most of the year that meant there was an API. There was, but it had grown one screen at a time, and the website, the staff screen and the assistant each answered the same question in their own way. So we spent the month on the layer under the loop.

What it is made of

• Every action has two doors: plan it, then do it. The first door takes what the customer seems to want and works out what the business already knows about it: which child, which service, whether they are covered, which dates are open, what the rules say. It answers with a question, a reason it cannot go ahead, or a quote. It remembers nothing. The second door takes the quote the customer said yes to, checks everything again, and does it once, however many times the yes arrives.
• The conversation is not a process. Nothing waits inside it. What is durable lives in the records of the business, a place held, a checkout waiting to be paid, a plan that is paused, and the assistant reads those like anyone else.
• One answer per question. What cancelling costs, when the next place is free, whether a plan covers this, what is on sale: each now has one answer the website, the staff screen and the assistant all read, and a test fails if a second copy comes back.
• The checks are written beside the action, and tested. Whether an action takes money or a place, whether it is recorded, whether a repeat is harmless: stated next to each one, and checked on every change rather than believed.

What it can do now

Mostly what the website could already do, reached through the same two doors: pause and resume a plan, cancel a season plan under its own policy, answer a waiting-list offer, update a child's details, change the number the assistant is reached on, see credits and packs, see and cancel an order, ask for a copy of your data. Anything that costs money or a place is quoted first and done only after the yes to that exact quote.

Two things are new to every surface. A customer with no account can make one in the conversation: a name, an email, a code to the number, and the link to set up a login goes to the email. And on the shared number, where nobody is a customer of anyone yet, a business owner can start a Bizzly plan from the chat, with the prices the pricing page shows and the checkout link after a yes.

What I would do differently

Everything above is a principle we arrived at by building. Starting again on Monday, three things would be different. None are technically difficult. Each is a way of settling up front what an interpretation is allowed to cause, rather than working it out as you go, and version four is, in the end, what you get by doing all three.

Write down what the AI is never allowed to decide, first

Before any code. Ours ended up as: it never decides whether someone is verified, what a price is, whether a purchase is allowed, whether there is space, or whether a reply is safe to send.

Write it before the first line of code, not after the first incident. It takes an afternoon, and it is the most useful document about the system: the only one that answers what this thing can do to a customer without anyone having to read the codebase. In version four that list is no longer a document. It is the set of checks every request passes on its way to becoming an action, and the AI is offered nothing that bypasses them.

Price every action by what it costs to get wrong

Here is the question this answers. When the assistant gains a new ability, which checks does it need? Answer that from scratch each time and you have the same argument again for every feature, and similar actions end up protected differently depending on who built them and when.

The answer is to price the action, not debate the feature. Every action the assistant can take falls into one of four bands by what a mistake costs. A wrong answer costs one more message. A wrong send costs money and some goodwill. A wrong booking takes somebody else's place. A wrong payment is real money and a dispute. Each band has a fixed set of checks, and each band keeps every check of the band above it and adds its own.

The payoff is that a new ability never needs its own debate. Moving a session to another day is a new feature. It lands in the third band, so it needs exactly what the third band needs, and nothing about that depends on who is building it that week. The same table also settles what the AI may do on its own judgement: anything in the first band, freely, since the cost of being wrong is one more message; anything in the last two, only after the customer has said yes to the exact thing quoted.

We arrived at these bands one capability at a time, which is why the same checks were moved around for months. Written down on day one it is an afternoon of work. In version four it is not a document at all: each tool the AI can call is marked with its band, and the gate in front of the tools enforces that band's checks.

Build the tools that let you see and rerun conversations, first

The conversation viewer, the simulator and the recorded history are not things you add to a working system. They are what make a system improvable at all. They belong before the second feature, not after the tenth. They are also what let us replace the map in a week: every archived conversation that had gone wrong became a test the new version had to pass first.

Where freedom helps, and where it costs

If you take one thing from this article, take this table. "How much freedom should the AI have" is the wrong question, because the answer is different for every decision.

There is a clean split here. Freedom helps when the cost of being wrong is one more message. It hurts when the cost of being wrong is a charge, a booking, or telling someone something they should not know. The second kind does not improve with a better AI model, because that class of failure is not about intelligence. It is about which component holds the decision. A judgement call belongs to the AI. A rule belongs in code, where it can be stated once and shown to hold.

What "agentic" turned out to mean

First, the plain answer: it worked. A parent messages at nine in the evening, in their own words, about a specific child in a specific class, and gets a correct answer. They can move a session, buy a pack, cancel one, ask what they are owed. The number of ways people phrase those things is endless and no menu was ever going to cover it. It handles operations a traditional chatbot could not attempt, and it does so against live data with real money involved.

So the goal was met. What I want to question is the word we have all agreed to use for it.

When people talk about agentic AI, the picture in the room is of the AI doing three things: working out what is needed, deciding whether it is allowed, and carrying it out. Autonomy is the headline. Version four hands it the first of those completely, and that is exactly the step that made it good. It never gets the second or the third, and that is exactly what makes it safe. The autonomy that made this work was over the conversation, not over the consequences, and I suspect the same split is what makes anyone else's work.

What made it work was a business whose processes were already pinned down and already reachable by something other than a web page. Availability, entitlement, what may be sold to whom, what a cancellation means: all of it defined, enforced in one place, correct before any AI existed, and exposed through an API that anything could call.

That last clause is not a technicality. A rule that exists only inside a screen is not a rule an assistant can use, and the gap between "we know our process" and "our process is callable" is where most of the real work sits. The AI supplies a natural language way in to processes that were already well understood and already addressable. It does the part that genuinely needs flexibility, which is understanding people, and it is kept away from the part that needs to be right.

That is not a disappointment, and I do not want it read as one. It is a precondition, and knowing it up front would have saved us months of looking for the sophistication in the wrong layer.

So the answer I would give now has three parts, and they are the three at the top of this article. We learned them in the wrong order, which is most of why it took four versions.

Make the process callable. That is the paragraph above, and it is the part nobody scopes. Availability, entitlement, pricing, payment and the rules of the business, reachable by something other than your own web pages. Until they are, an assistant is a second copy of the business; once they are, it is a channel into the one you already run.

Keep the AI on interpretation and decisions. For two versions I asked how much decision-making you can hand over before it becomes a mess, and each answer was more conservative than the last: a map of routes, then a map with rails. The mess never came from the AI deciding too much. Version one was a mess because the AI could act on what it decided with nothing in between, and versions two and three improved slowly because code was doing the deciding the AI was better at. Handing the whole conversation back to it, once nothing it decided could reach a booking or a payment without passing the checks, made things better and not messier. The line between the two is reversibility. What the customer meant, what to look up, what to say, what to ask next: cheap to get wrong, and the AI's. Money, a scarce place, a sentence that cannot be unsaid: a rule with a name and a test, and the customer's own yes in front of it.

Write the policy down, and make that callable too. I used to think some decisions with consequences had no rule behind them, because the whole value was judgement: who should be offered the place that just came free, which of two families to move when a session fills. That was wrong. Those are policies. A business decides that a freed place goes to the waiting list in order, or to members first, and once that is written down the system applies it and the assistant explains it. Every time we thought we had found a judgement call, we had found a policy nobody had written down yet, and writing it down was the work. What is still unwritten is where this article ends.

Eight lessons

Eight months, four architectures, and these are what I would hand someone on day one. The first three are the short version at the top; the rest are what it took to learn them.

1. Make the process callable before you write a prompt. The AI was the cheap part. What actually decided when we could go live was making the business rules consistent, and reachable by something other than our own web pages. That is the work sponsors under-scope, every time, because it does not demo.

   It is also the only part of this with a growth curve rather than a maintenance curve. The same entitlement checks, purchase rules and checkout the assistant uses are the ones the website and the staff screen use, so every future channel inherits them for nothing. Which gives you a planning question you can answer this week: whatever is not yet behind that layer is the hidden cost of your next AI project. October was that month for us, and it is the section above about the layer under the loop.

2. Let the AI interpret and decide. Never let it execute. It is better than any rule I could write at two things: understanding what somebody meant, and working out what ought to happen next. It is worse than any rule at a third: doing the same thing the same way twice. That third one is the property a business process cannot survive without. Hand over the judgement. Keep the doing.

3. Write the policy down, and make it callable. Before any code, list what the AI may never decide: who someone is, what a price is, whether a purchase is allowed, whether there is space, whether a reply is safe to send, who gets a place that comes free. Every item on that list is a policy the business owns, and every hard case we met was a policy nobody had written down yet. Written down and reachable by a call, it is a rule the system applies and the assistant explains. Left in someone's head, it is a judgement the AI will be asked to make. Write it first and it shapes the architecture. Arrive at it later and you are retrofitting it into decisions already taken.

4. Budget for one structural rewrite, and stop treating it as failure.Four architectures in eight months, and every change was forced by a problem we had not anticipated rather than chosen from a roadmap. You cannot specify these risks up front, because they live in what real customers type. So plan the rewrite as a cost of the programme. A team that has not rewritten anything has probably not learned anything yet.

5. Put your strictness where the consequence is, then check who it catches.Asking someone to prove who they are before they have learned anything protects the system and loses you the customer. Move it to the moment something becomes irreversible, and rank every action by how easily it can be undone.

   Then the part most teams skip. A control tuned only against the abuse case will catch good-faith customers, and it will do it at the worst possible moment. Every new guard needs its false positives reviewed before it ships, not just its false negatives.

6. Assume the damage comes out, not in. Almost all published AI safety advice concerns what people send you. The expensive failures are fluent, confident replies that are wrong about your business, and nothing upstream catches them because the customer asked a perfectly reasonable question.

   Worse, the context that makes an answer correct does not stay proven. Which business this conversation is about, and who is asking, must be re-established by every capability you add. A search or a pricing path built in month six does not inherit it from one built in month two.

7. Verification is a permanent second engineering track, not a phase. You cannot test your way to confidence, because the same input does not reliably reproduce the same failure. Conversation capture, replay and checks that run over real traffic are not a one-off investment you pay down. They have to grow with the system. Fund them once and you will slowly lose the ability to say whether the thing still works.

8. Once the rails are real, give the loop its freedom back. A map of routes is what you build when you do not yet trust the checks around the AI. It earns its keep for a while, and then it becomes the place where language you should not be writing in code goes to live. The test for whether you still need it is whether any check depends on it. When the answer is no, the map is only deciding what the customer meant, and the AI was always better at that.

One last observation, for anyone hoping to wait this problem out. Over these eight months the models got materially better, and version four does hand the assistant more of the conversation than any version before it. Its authority did not grow with it. Every extension of what it may do ended up narrower than it started, never wider. Model progress moved how much reading and deciding we trust it with. It did not move the ceiling on doing, because that constraint was never the model's intelligence.

Which is why the line is not mysterious for anything the assistant does: it is exactly the set of actions that already have a callable, tested, business-scoped rule behind them, and it moves as fast as you shorten that list. That is a punch list, not a philosophy question, and it is fundable.

The one question I still cannot answer is the unwritten part of the third lesson. Every business runs on policy it has never written down, and the last of it is the exception: the one-off an owner grants by feel. The place held an extra day for a family who always pays. The session moved as a favour. Today the assistant hands those to a person, and the person decides, which is the right answer and also the boring one. Whether the assistant should ever propose the exception itself, and whether an owner's answers should quietly become the next written policy, is the part we will find out by trying, somewhere a wrong answer costs nothing but a second message.

Questions people ask