When someone shows off a working AI agent, what you see is the conversation: the question goes in, the answer comes out, sometimes the agent checks a system and comes back with the right data. It looks like everything happens inside the AI model.
It doesn’t. Behind any agent that does real work in an operation, there’s a layer that decides what it remembers, what it can do, where it has to stop and how each step gets recorded. That layer is called orchestration. For anyone hiring, understanding its basics is the best way to tell a slick demo from an agent that can handle the day-to-day.
The model thinks, orchestration steers
On its own, an AI model takes in text and returns text. It has no memory between conversations, doesn’t access your ERP, doesn’t know which actions require approval and keeps no history of anything. All of that is the job of the system around it.
An analogy helps. Think of a very capable new hire. They can reason, but they need access to systems, a folder with the customer’s history, rules about what they can decide on their own and a manager who checks their work. Orchestration is all of that, built in software.
In practice, it handles four things: state, tools, limits and logging. When an agent gets something wrong in operations, the cause is almost always one of these four pieces, not the model’s intelligence. It forgot what the customer said, used a tool it shouldn’t have, went past a limit nobody configured or did something nobody can reconstruct afterwards. Each one deserves a closer look.
State: what the agent knows right now
State is the agent’s working memory. Which step of the process it’s on, what the customer has already said, what data has already been looked up, what’s left to do. Without state, the agent asks again for things already answered, loses the thread when a conversation is interrupted and can’t resume a process that spans days.
Imagine an agent following up on an overdue invoice. The customer replies on Tuesday saying they’ll pay on Friday. The following Monday, the agent needs to know about that promise, check whether the payment came in and decide the next step. That only works if the state was stored outside the model, somewhere reliable.
Good questions for whoever will build it:
- Where is each case’s history stored, and for how long?
- If the agent crashes in the middle of a task, does it pick up where it left off?
- Do two messages arriving at the same time for the same case turn into two duplicate actions?
Tools: what the agent can do
Tools are the actions the agent can take: look up an order in the ERP, create a deal in the CRM, send a WhatsApp message, generate an invoice. Each tool is a door into one of your systems, and each door has to be designed carefully.
The key point is giving the agent only the tools the task requires, with minimum permissions. An agent that answers order status questions needs to read orders. It doesn’t need to cancel orders, change prices or see other customers’ financial data.
How the tool behaves when something goes wrong matters too. If the ERP is slow to respond, does the agent retry? How many times? Does it alert someone? These decisions look like technical details, but they’re what prevents an order being saved twice or a message going to the wrong customer.
Limits: where the agent has to stop
A limit is the rule that says how far the agent goes on its own. Some common examples:
- A discount above a set amount needs a person’s approval.
- Any mention of a lawsuit goes straight to legal.
- Refunds only after someone in finance checks them.
- A message to a customer with a serious complaint is reviewed before it goes out.
- If the agent isn’t sure, it asks or hands off, instead of guessing.
These limits need to live in the orchestration code, not just in the model’s instructions. An instruction written in text is guidance the model follows most of the time. A rule in the system is a barrier it can’t get past. For what really matters, you want the barrier.
Whatever can’t go wrong has to be a system rule, not a polite request to the model.
Logging: proving what happened
Everything the agent does needs to be recorded: what it received, what it looked up, what it decided, which action it took and who approved it, when there was an approval. That record serves three purposes.
- Investigating mistakes. When a customer complains, you need to see exactly what the agent did and why.
- Improving the agent. The cases it handed to a person or where it was corrected show where to adjust.
- Accountability for personal data. Privacy laws like GDPR require companies to know how they process people’s data and to be able to demonstrate it. An agent with no logs leaves you without an answer.
Logging needs care too. It shouldn’t keep more personal data than necessary, and access to it should be limited to the people who need it.
Why this should weigh on your decision
Two agents can use the same AI model and behave completely differently in operations. Almost all of the difference is in the orchestration. That’s why comparing vendors by the model they use says little. Comparing them by how they handle state, tools, limits and logging says a lot.
When you’re evaluating who will build your agents, ask to see:
- A real case being resumed after an interruption.
- The list of actions the agent can take and the permissions for each.
- Where the approval rules live and who can change them.
- The full log of one interaction, from start to finish.
- What happens when an integrated system goes down.
If the answers to these questions are vague, the agent may well work in the demo. But your operation will discover the gaps the worst possible way, with a customer on the other end.
The AI model gets better on its own with every release. Orchestration is what you need to make sure someone built well.