The question comes up in almost every conversation, usually as "should we be doing something with AI?" The honest answer is that the question is too broad to answer. Over the past few months I read AI Snake Oil by Arvind Narayanan and Sayash Kapoor (Princeton University Press, 2024), and the book supplies exactly the vocabulary those conversations lack. Its core point: "AI" is an umbrella term, like "vehicle". A bicycle, a truck and a rocket are all vehicles, but nothing useful can be said about all three at once. The same goes for AI. A chatbot and the algorithm a bank uses to assess a loan are both called AI, but they work differently, are used differently and fail differently.

Below is the classification we have used since, along with the trade-offs we make in practice. Written for anyone who wants to use AI where it works, and wants to know where it stops.

First question: which AI are we talking about?

The book distinguishes three kinds. Predictive AI uses historical data to forecast an outcome: who will be a good employee, which customer will leave, which patient will deteriorate. Generative AI produces new text, images or code from a prompt. And AI for content moderation, which platforms use to filter posts. We'll leave the last one aside here; the first two we encounter daily.

That distinction determines whether an application has a chance of working, before you spend a euro on it.

Predictive AI: where most of the snake oil lives

The authors' claim is sharp: predictive AI about human behaviour does not work the way it is sold, and probably never will. The maths is not the issue. People cannot be predicted from last year's dataset. A model learns patterns from the past and works as long as the world resembles that past. Once the context shifts, reliability shifts with it, and you only notice afterwards.

Two examples from the book stick. A model predicting pneumonia risk concluded that asthma patients were lower risk. In the data, that was true: asthma patients were sent straight to intensive care and survived more often as a result. The model learned the effect of the care, not the risk of the disease. And at a US health insurer, staff were required to follow AI decisions on claims; of the denials that customers appealed, the large majority were later overturned.

We get this question too. "Can we predict which requests will go wrong?" or "which customers will cancel?" The line we draw is human behaviour as the predicted outcome. How much stock you need in November, when a machine needs maintenance, how revenue moves with the season: those are predictions about systems with enough regularity, and we build them in Power BI and AI Builder. Who will cancel, who will commit fraud: those are predictions about people. We only deploy those as a signal, with a human making the decision and treating the model as one input among several.

Generative AI: it works, just not like the demo

On generative AI the authors are more positive, with caveats. A language model produces the statistically most likely text for a given prompt. That yields surprisingly useful results, but the model has no notion of true or false. It gives an answer that sounds right, and usually it is. Usually.

Where it delivers value in our practice: summarising, rewriting, classifying, retrieving information from your own documents, drafting something a person then finishes. The common trait: a mistake is cheap, and someone looks at the output before it has an effect.

Where it disappoints: facts without a source, arithmetic, reasoning about things not present in the supplied context and consistency over time. Organisations underestimate that last one the most. We wrote earlier about an AI Builder prompt that dropped from near one hundred percent accuracy to thirty or forty, without a single change on our side. The underlying model had been updated, and our prompt responded differently to it. If you don't measure output periodically, you find out when someone complains.

Why the takeover story is snake oil, on today's architecture

Now the point I hold a firm opinion on. The narratives about AI outgrowing humanity or escaping human control, I consider hype. With one boundary up front: my argument is about the architecture we have today. The book devotes a chapter to it and the arguments are convincing.

What is that architecture? The language models behind ChatGPT, Microsoft 365 Copilot and Copilot Studio are systems that, given a piece of text, compute the most likely continuation. The latest generation does that better than the previous one, including on reasoning and coding tasks; that progress is real. But it remains the same kind of system. After training, the model is fixed. It learns nothing lasting from the conversations it has; a memory feature or a database next to it stores facts, the model itself does not change. It has no interests of its own. And it does nothing as long as nobody calls it. A model that isn't called exists as a file on a disk.

Goals are where it gets more complicated. Apollo Research and Anthropic showed in test setups in 2024 and 2025 that models can, within a task, display behaviour that looks like self-preservation or deception, for instance by hiding unwanted behaviour when they infer they are being tested. That behaviour has only been seen in artificial setups that supply the goal themselves; outside such tests it has not been observed. There is no goal the model carries from one conversation to the next of its own accord. What is in there are the preferences that training put in, and those are the same in every conversation. That is learned behaviour; there is no will behind it. And it is why you don't give an agent permissions it doesn't need.

An "agent" changes none of the above. An agent is a control layer that people build around such a model: a loop that calls the model again and again, with access to tools and permissions that people grant. If an agent does something stupid, it is because someone built that loop and gave it that permission.

The doom scenarios assume that progress from "useful chatbot" to "superintelligence" is a straight line you can extend by adding more data and more compute. But what a genuine general intelligence requires is simply not part of this architecture. A child falls down the stairs once and does it differently from then on. Nobody retrained that child; it understood what height is and what falling does, and it kept that. A language model can have read a million texts about falling and still not know what height is. It knows which words usually appear near "height". As long as a system doesn't understand what it is dealing with, and doesn't learn lastingly from its own experience without a human running a training round, it is not AGI, no matter how much compute you throw at it. More of the same yields a better text predictor.

And I think this is the most important part: the fear and the hype come from the same corner. If your product can take over the world, it is apparently powerful enough for your customer service. The same presentations in which Microsoft 365 Copilot will change your organisation come from a company that has put billions into OpenAI and therefore has a stake in the image of a technology that can do anything. That is marketing, and it is the same snake oil in different packaging.

With a fundamentally different architecture, a system that does understand what it deals with and does learn lastingly from what it experiences, the questions are different. I don't rule that out. But that architecture does not exist today, and nobody can say whether or when it will. So we plan on what exists.

What this means for Microsoft 365 Copilot and Copilot Studio

The book's classification also helps to look at Microsoft's Copilot offering with a level head, and to decide where to start and where not to.

Microsoft 365 Copilot is generative AI over your own mail, documents and meetings. It does what generative AI does well: retrieve, summarise, draft. That is where the value is, and it is concrete: less time spent searching and rewatching, a usable draft sooner. Two limits we keep pointing out. Copilot only sees what the user is allowed to see, so your permission structure determines both the quality and the risks; anyone who can see too much now gets it neatly summarised as well. And if your SharePoint is a mess, you get a well-written summary of a mess. You fix both before the rollout, not after.

Copilot Studio goes a step further: you build agents that consult knowledge sources and perform actions through connectors and flows. This is where the book's distinction becomes practical. An agent answering questions from the employee handbook or the product documentation is low risk: wrong answer, human corrects. That is where you start. An agent that approves a leave request or changes an order on its own needs limited permissions and a human confirmation on the step that matters. That can be done, but it is design work. And an agent that uses a language model to "predict" whether a customer will pay, we don't build; that is predictive AI in a generative wrapper, with the same problems.

The rules of thumb we apply before building anything:

  • Is it about predicting what a human will do? Then only as a signal, with a human deciding.
  • Does the agent get permissions? Limit them to what is needed and require confirmation on the irreversible step.
  • Measure the output periodically against a fixed test set. What works today does not automatically work in three months.

The risk we run into at clients is rarely spectacular. It is a fluently worded answer that nobody checks anymore. That is less exciting than a robot uprising, and it is already happening.