There is a commercial reason most AI vendors never publish an article like this. A generative build is a larger engagement than a gradient boosted model on data you already have. Recommending the smaller piece of work costs revenue.
We publish it anyway, because the alternative is delivering an expensive system that underperforms a simpler one, and that costs more than the invoice was worth.
Four tests that usually decide it
Before scoping any system we work through the same four questions. They are not exhaustive, but they resolve most cases within an hour.
- Is the input language, or is it fields? If the data is rows and columns with stable semantics, a language model is being asked to do statistics through a text interface.
- Does the output need to be explainable to a regulator or a customer? A model that produces a decision boundary you can inspect is worth a great deal in regulated environments.
- Is someone actively trying to defeat it? Adversarial problems need fast retraining loops and stable, auditable features.
- What is the cost per event, multiplied by volume? A cent per call is invisible at a thousand events a day and ruinous at ten million.
Structured prediction is not a language problem
Churn prediction, demand forecasting, credit risk, lead scoring, price optimisation, predictive maintenance. These are the problems machine learning solved well before the current wave of tooling, and gradient boosted trees still beat language models on them in accuracy, latency and cost, usually by a wide margin.
The reason is simple. Your features are already numeric and already meaningful. Serialising them into a sentence so that a model can re-derive relationships it has to infer from text is a lossy detour. A model such as XGBoost or LightGBM trains in minutes on a laptop, produces feature importances your analysts can argue with, and answers in single-digit milliseconds.
One retail platform we built runs recommendations and demand prediction under 100ms end to end, with demand forecasting accuracy improving 62%. That system uses classical methods for the prediction layer precisely because the latency and cost budgets left no room for anything else. measured · retail-platform · prod
If your features are already numeric and meaningful, asking a language model to re-derive them from a sentence is a lossy detour.
Adversarial problems need a fast retraining loop
Fraud detection, abuse and spam classification, trust and safety on a marketplace. These share a property that changes the engineering: an intelligent opponent adapts to your defences, so the data distribution moves deliberately rather than incidentally.
What you need is a model you can retrain quickly on fresh labels, monitor for drift, and explain when a customer disputes a decision. Classical classifiers on engineered features do all three well. A language model does none of them well: retraining is slow and expensive, the decision path is opaque, and per-event cost at fraud-screening volume is prohibitive.
Language models still have a place at the edges of these systems, summarising a case file for a human reviewer or drafting the customer communication. That is a different job from scoring the event.
If the process is deterministic, write the process
A surprising share of what gets pitched as agentic AI is a scheduled job with three conditionals. Move records from system A to system B, apply a threshold, notify someone if it is breached.
Wrapping that in a language model adds cost, latency and non-determinism to something that used to be reliable, and it makes the failure modes harder to diagnose. When a deterministic job fails you read the log and see what happened. When an agent fails you read a trace and try to work out why it chose what it chose.
Our rule: use a workflow engine where the path is stable and known, and reach for an agent only where genuine judgement is needed at each step. We self-host workflow automation for exactly this reason, and a large fraction of the automation we deliver contains no generative model in the hot path at all.
Fixed-format extraction has a cheaper answer
If you are pulling the same eight fields off the same invoice template a hundred thousand times a month, document AI services and template-based extraction are dramatically cheaper and more accurate than an LLM call per page. They also fail predictably, which matters when you are reconciling financial records.
The calculation changes when the formats are heterogeneous and unpredictable, when the content is prose rather than fields, or when you need reasoning across several documents at once. That is where a language model starts earning its cost.
Where a language model clearly does win
This is not an argument against generative AI. It is an argument for putting it where it is strongest:
- Open-ended questions over documents, where the query cannot be anticipated and the answer has to be assembled from prose. This is what retrieval augmented generation is for.
- Multi-step work across tools where the path varies case by case and the judgement at each step is linguistic rather than numeric.
- Language transformation: summarising, translating, drafting, restructuring, changing register. Nothing else comes close.
- Long-tail classification where you have hundreds of categories and not enough labelled examples per category to train a conventional classifier.
- Domain-specific generation after fine-tuning, where tone, vocabulary and format matter and you have proprietary data to align on. One legal deployment reached 91% citation accuracy with zero hallucinated citations after alignment. measured · legal-ai · prod
Most real systems end up hybrid
The interesting answer is usually not one or the other. A support platform might classify intent with a small fast classifier, retrieve with hybrid search, generate the reply with a language model and score sentiment with a conventional model to decide escalation. Four different tools, each doing what it is best at, with cost concentrated where it produces value.
The architecture question is not "should we use AI". It is "which part of this problem is language, and what is the cheapest correct tool for everything else". Getting that split right at the scoping stage is worth more than any downstream optimisation.
Key takeaways
- Tabular prediction, forecasting and risk scoring belong to classical machine learning on accuracy, latency and cost.
- Adversarial problems need fast retraining and explainable decisions, which favours conventional classifiers.
- A stable, known process should be a workflow, not an agent.
- Language models win on open-ended questions over prose, genuine multi-step judgement and language transformation.
- Most production systems are hybrids, with generative cost concentrated where it earns its place.