A colleague asked me last year which model to use for a literature synthesis — fifty-odd papers, extract the methods, find where they disagree. We spent an hour comparing options before doing any actual work. That hour was mostly wasted, and I have since developed a fairly cynical view of model-comparison content, including the kind I am about to write.
So let me start with the caveat rather than bury it. Published benchmark scores tell you almost nothing about how a model will behave on your task. This is the same problem I deal with in medical imaging: a number produced on a curated evaluation set, by the people who want the number to be high, does not transfer. The only comparison that matters is the one you run on your own work, and it takes about twenty minutes to run.
With that said, there are real and fairly stable differences in how the major models behave, and knowing them saves you from testing blindly.
The Shape of the Landscape
In 2023 there was GPT-4 and then a considerable gap. By 2025 that gap has closed to the point where, on most ordinary tasks, the top models from OpenAI, Anthropic and Google are hard to tell apart in blind comparison. Which one "wins" a given test usually says more about the test than about the models.
The differences that survive are differences of temperament: how a model handles ambiguity, how it behaves when it does not know something, how well it holds onto a long list of constraints, how it fails.
GPT-4o
GPT-4o remains the default for most people, largely because ChatGPT got to the mass market first and the surrounding ecosystem is the most built-out. The model is strong across the board and genuinely excellent at agentic work — chaining tool calls, browsing, running code, holding a multi-step task together.
The code interpreter and data analysis features are the part I actually use. Handing it a CSV and asking for a distribution plot and a sanity check on the column types is faster than writing the pandas myself, and it catches things I would have missed.
Its characteristic failure mode is confident wrongness. Every model hallucinates; GPT-4o hallucinates in a register that sounds like an expert who has already checked. When it produces a specific figure, a citation, or a claim about an API's behaviour, treat it as a hypothesis. I have been burned by this on exactly the kind of technical detail that looks too mundane to verify.
Claude
Claude's distinguishing property is how it handles long documents and dense instructions. The large context window is the headline feature, but the thing that makes it useful is that the model tends to actually use the middle of the context rather than anchoring on the beginning and end. For reading a full protocol document, a contract, or a codebase before answering, this is the difference between a useful answer and a plausible one.
It also follows multi-part constraints more reliably than the alternatives in my experience. If I have specified tone, structure, what to include and three things to avoid, Claude is the model most likely to still be honouring all six constraints at the end of a long response.
The property I value most is harder to benchmark: it is more willing to say it does not know. Given a question at the edge of its knowledge, it is more likely to flag the uncertainty than to fill the gap smoothly. For anything I am going to act on, a model that surfaces its own uncertainty is worth more than a model that is marginally more accurate on average.
For coding, Claude and GPT-4o are close enough that the choice is preference. Claude has an edge on writing new code from a specification; I find GPT-4o slightly better at reasoning about an unfamiliar error in someone else's stack.
Gemini
Gemini's structural advantage is Google: current web information, Workspace integration, and genuinely native video understanding rather than video-as-sampled-frames. If your work lives inside Google's ecosystem, the integration is not a marketing point, it is a real reduction in friction.
The Flash tier deserves more attention than it gets. It is fast and cheap enough to be practical for high-volume pipelines — classification, extraction, routing — where you are making thousands of calls and do not need frontier reasoning on any individual one. That is a different product category from "best model", and it is the one that determines your API bill.
Where Gemini has tended to lag is sustained multi-step reasoning and prose quality. It is technically competent and stylistically generic, which matters for some tasks and not at all for others.
Open Weights
The open-weight story in 2025 is better than most people outside the field realise. Llama 3's larger variants, Mistral's models, and DeepSeek's reasoning models are genuinely usable for serious work, and the tooling — Ollama for local, vLLM for serving, Groq for fast hosted inference — has stopped being a research project.
A 70B-class model today is comfortably past the GPT-3.5 era and handles most everyday tasks without you noticing the difference. Where the proprietary frontier still pulls ahead is the long tail: subtle reasoning, unusual instruction combinations, tasks where the model needs to notice that the question itself is wrong.
The case for running open weights is strongest when you cannot send the data anywhere. That is my situation more often than not — patient-adjacent data does not go to a third-party API, and no amount of contractual assurance changes that calculus. The other strong cases are fine-tuning on domain data and high-volume inference where per-token pricing dominates your costs.
The case against is that you are now operating infrastructure. Somebody has to own the GPUs, the serving stack, the updates and the outage at 2am.
How I Actually Choose
The heuristics below are mine and they are provisional, which is the point.
For long document work and anything where instruction-following matters, Claude. For agentic tool use and quantitative scratch work, GPT-4o. For high-volume extraction pipelines, a Flash-tier model or a hosted open-weight model. For anything involving data that cannot leave the building, self-hosted, and the conversation ends there.
But the heuristic that actually matters is this: build a small evaluation set out of your own real tasks — ten or fifteen examples with outputs you can judge — and run it whenever you are considering a switch. It takes an afternoon to build and it will outlast every model generation. Every benchmark chart you read was built by someone with an interest in the outcome. Yours is not.
And keep your integration thin enough to swap the model underneath. The competitive ordering has changed roughly every six months for three years. Anything in this article that reads as a durable fact about model rankings should be assumed stale within two releases.
Tags
Taresh Sharan