I recently ran an internal session for a semi-technical audience with the title “What makes an agent, an agent?” When I proposed the subject, the concept was met with real enthusiasm, which I took as a sign that people would welcome a plain answer to a question they are assumed to have settled already.
The word “agent” is used so loosely at the moment that people who build these systems will each give you a different answer about what it means. Anyone asked to sign off a product sold as an agent is often not sure which of those answers they are getting.
I began the session with the model, because it sits at the heart of every agent, and the ones on my laptop turned out to be quite sure of several things that are no longer true.
A brain in a jar
A large language model is a file, and on my laptop I have two Qwen models, each a few gigabytes on disk, which run through Ollama. When a question sets one thinking for a minute or more, the fans spin up and the chip climbs past ninety degrees while it works. Everything stays on the machine, and equally nothing new reaches the model from outside, which is the point to hold on to.
The model knows what it was shown during training, up to a date, and after that date it knows nothing at all. If you drew its knowledge against time you would get a line rising to the cut-off and then running flat, and everything to the right of the cut-off is a gap that the model cannot see. It has no clock and no way of looking anything up, so it cannot tell you the date or where it is. In the picture I used on the slide it is a brain in a jar, capable of a great deal of reasoning and entirely dependent on whatever was put in the jar with it.
The same limit applies to every model, however large or capable, because it comes from the way models are trained, so it describes what a model is and is not a fault in this one.
Four questions for a brain in a jar
The quickest way to make the point is to ask the model things it cannot know. I asked it the time, and it told me, politely, that it had no access to real-time data and suggested I check my phone. I asked it for the weather, and it apologised and recommended the Met Office.
Those two are honest failures, and the model is right to decline. The next two matter more, because in both cases the model answered anyway.
I asked it about the latest iPhone, and after nearly ninety seconds of thinking it told me, in some detail and with complete confidence, that the latest model was the iPhone 16, announced in September 2024. That was true on the day the model’s training stopped and it has not been true for some time, but nothing in the answer gives you any hint of that.
Then I asked the larger of the two models for the last six UK prime ministers, with the instruction that anyone who served fewer than fifty days should be marked with a lettuce. It listed Rishi Sunak as the current prime minister, which he had not been for two years, left Keir Starmer out altogether and went back to Gordon Brown to make up the six. It gave Liz Truss sixty-two days, from a July start date she never had, which is wrong by nearly a fortnight and denied the lettuce its moment. Watching the reasoning scroll past in the terminal was instructive in itself: “wait, but that’s only five”, “maybe I’m missing some”, and round it went, with the fan at four thousand revolutions a minute, thinking hard about a question it had no way of answering.
So the models were wrong about something after their cut-off, which is forgivable, and wrong about something well before it, which is less so, and they delivered both with the same fluency it uses when it is right. If you take one thing from the session, let it be that fluency and accuracy are separate things, and only one of them shows in the answer. I have written before about treating a model’s confidence as noise, and this is the same point made with a lettuce.
Same question, different answer
The second thing people need to know about a model matters just as much as the cut-off. A model produces its answer one word at a time, and it picks each word from a range of likely candidates according to how probable each one is. That is what people mean when they call a model probabilistic, and what you see from the outside is that it is non-deterministic, because the same question asked twice can come back with two different answers.
In the session I made the point with pictures of a cat, because a picture shows the variation more plainly than a paragraph of text. I gave Claude Opus 5 and Claude Sonnet 5 the same prompt three times each, and got six different cats, one of which appears to be sitting in its own teacup.
When I asked two image models, Nano Banana and ChatGPT, for a cat, the results were further apart still, and that is something to keep in mind the next time somebody proposes swapping one model for another and expects the system to behave as it did before.
A different cat each time is harmless, but the same behaviour applies when the output is a piece of code, a summary of a contract or a decision about a customer. Turning the randomness down to its lowest setting does not remove it entirely either, and researchers at Thinking Machines found that “even when we adjust the temperature down to 0 (thus making the sampling theoretically deterministic), LLM APIs are still not deterministic in practice”.
The first consequence is that one good answer tells you very little, so a test that passes once has not shown that the agent is reliable, and anything you depend on needs checking over many runs. The second is the one I raise most often, which is whether the problem needs AI at all. A great deal of what gets proposed as an AI problem turns out to be a rules problem, and if the steps can be written down, ordinary automation or business logic will do the job the same way every time, leave an audit trail and cost a fraction as much to run. I’d suggest starting there, and bringing in a model for the parts of the problem that need judgement over language or untidy input.
Hosted, local, open weight
Everything above applies equally to the frontier models you reach over the internet. GPT, Claude, Gemini and the rest are the same kind of thing at a much larger scale, running in somebody else’s data centre, with their own cut-off dates and their own flat line to the right of it. If ChatGPT or Claude can tell you this morning’s news, that is because the app around the model has given it a search tool, which is the rest of this post in miniature. The difference between a hosted model and a local one is where the jar sits, and for some conversations, particularly ones involving data you want to keep inside the building, that difference is the whole decision.
Vendors often use “open weight” and “open source” as if they meant the same thing, and the difference matters. When Meta, Alibaba, DeepSeek or Mistral release a model, you get the weights, which are the file I have been describing, and a licence to run it, while the training data and the recipe stay with them. With the weights you can run and fine-tune the model, but rebuilding it or fully accounting for what went into it needs the data and the recipe, and anyone doing supplier due diligence should know which of those they are getting.
So what is an agent?
The dictionary is a good place to start, and it defines an agent as a person or thing that acts, or has the power to act. A brain in a jar can only produce text, once, in response to whatever it was given, and then it stops, so by that definition the model is one part of an agent and the rest has to come from somewhere else.
An agent has three working parts, and I would encourage anyone evaluating a product with “agent” in the name to look for all three, along with the two things that surround them.
The first is the reasoning, which is the model, and although it is the part everyone talks about, from here on it is the least interesting.
The second is tools, which are things the model can call on when it needs them, such as a clock, a web search, a database query, a shell that will run code or an API into your order system. The model is given a description of each tool and decides when to use one, then asks for it to be run and receives the result back as text. Give either Qwen model a clock and a search tool and the lettuce question stops being a memory test, because the model can simply look the answer up. Tying a model’s answers to sources you trust in this way is called grounding, and it is the word to listen for when a supplier explains how their agent stays accurate. It narrows the room for invention, although the model still decides what to make of what it reads. That is the entire difference between the failing session above and a useful one, and the model itself has not changed at all.
The third is the loop, because a single call to a model with a tool is only a lookup. An agent calls a tool, reads the result, decides what to do next and calls another, carrying on until it judges the task complete or something stops it. Each turn of that loop is a decision the model is making about what to do, and the number of turns it takes before a person sees anything is the practical measure of how autonomous the thing is.
Around those three sits what I called the harness in the session, which is the software that holds the tools and runs the loop, and which also keeps whatever memory the agent has. The model keeps nothing from one call to the next, so even within a single conversation the harness sends the whole exchange back to it every time. If your agent appears to remember last week, that is the harness writing things down and reading them back. The notes it keeps are data that your organisation is now responsible for.
The other thing that surrounds them is what starts it, and although a prompt from a person is the obvious trigger, an agent can also start from a scheduled task, an incoming email, a change to a record in a database, a call to an API, or another agent finishing its own work. Once the trigger is something other than a person typing, you have a system that acts without anyone having asked it to on that occasion, and that is a different thing to govern from a chat window.
If you want the whole thing in one sentence, the one I put on a slide was that an AI agent is a language model that can use tools, running in a loop until it finishes a job. Simon Willison arrived at almost the same wording in September 2025, when he wrote that “an LLM agent runs tools in a loop to achieve a goal”.
AI agent, or agentic AI?
The two terms get used interchangeably, but one is a noun and the other is an adjective, and keeping them apart turns out to be useful.
An AI agent is a specific system, built from the parts above: a model, a defined set of tools, a loop, a trigger and, usually, some memory. Because each one is a distinct system you can count them, list what each one is allowed to touch, name the person who owns it and switch it off. When somebody says “we have deployed an agent to triage support tickets”, that is the sense they mean, and the questions at the end of this post apply to it directly.
Agentic AI, on the other hand, is a property, and it describes any system in which a model is allowed to decide what to do next and then act on that decision. That covers a single agent, and it also covers a workflow in which a model chooses which of several fixed steps to run, or a pipeline in which several agents hand work to one another. The word tells you that somewhere in the system a model has been given a degree of autonomy, and you have to ask separately how much.
That is why I find it useful to treat “agentic” as a scale, and the two ends of it look very different. At one end is a model with a single tool, where a person approves every step and reads every result, and some people would not call that an agent at all. At the other end is a process that runs on a schedule, holds write access to production systems, takes as many turns of the loop as it needs, and reports afterwards, if at all. Both get described as agentic, and both may be built on the same model. The difference between them is the tools, the trigger, the number of turns, and who is watching, and those are the things that need writing down.
So when the term arrives in a proposal, translate it before you respond to it, remembering that “agentic” only tells you there is autonomy in there somewhere. The questions that follow are how many agents, doing what, with access to what, started by what, and watched by whom.
Why this matters for leaders
When a supplier or an internal team tells me they are deploying agents, the model they have chosen is the last thing I ask about. The questions that matter are all about the harness, and they map neatly onto the anatomy above. They are also the same questions you would ask about any system that holds credentials and runs on a schedule, and a model in the middle makes them more urgent, because the model is just as confident and fluent when it is wrong about Liz Truss as when it is right.
What this means in practice
Ask whether it needs a model at all. If the steps can be written down as rules, ordinary automation will give the same answer every time, and a model belongs only in the parts of the problem that need it.
Separate the model from the agent when you evaluate anything. The model supplies the reasoning, while everything that lets it act belongs to the harness, which is where your risk and your governance live.
Assume the model is out of date. Design so that anything time-sensitive or factual comes from a tool, since the model has no way to check its own memory, and test it more than once with questions you already know the answer to.
List the tools before you approve the agent. Each tool is an access grant, so the list of tools is the list of things that can go wrong. Treat it the way you would treat the permissions on a new service account, because that is what it is.
Know what starts it. A prompt, a schedule, a message, a data change and another agent each pose a different governance problem, and a system that acts on its own schedule changes who is accountable for a given run, so write down which triggers apply.
Set the loop length on purpose. The number of turns before a person sees the result is how much autonomy you have granted. Record that decision where people can see it, so that the limit is one somebody chose.
Treat the memory as data. If the harness keeps notes, they need an owner, who decides who can read them and will be the person asked to delete them when the time comes.
The question worth asking
What matters is what you have plugged the model into, and whether the person who decided would recognise the list of tools and triggers if you read it back to them. If the answer is clear, you have an agent you can govern, and if it is unclear, you have a brain in a jar with the lid off, so you should probably find out what it can reach.
If you are working out how to govern agents in your own organisation and would like to compare notes, get in touch.