In March 2024, Cognition published a demo of Devin, described as an autonomous AI software engineer. It picked up a task, planned, wrote code, ran it, debugged its own failures and shipped. The launch post reported a resolve rate on the SWE-bench benchmark several times higher than previously published systems, and for a week it was all anyone in the field talked about.
Independent evaluations that followed were considerably less flattering than the demo. That gap — between a curated demonstration and what the thing does on your work — is the defining feature of agent technology, and two years on it has narrowed without closing.
I want to write about agents from the position of someone who builds systems that have to work rather than demo, because the engineering reality is more interesting than the pitch.
The Actual Definition
An agent is a model in a loop with tools and state. It observes, decides on an action, executes it, observes the result, and decides again, until it concludes it is done or something stops it.
That is the whole idea. Everything else — planning modules, memory hierarchies, multi-agent orchestration — is structure built on top of that loop to make it survive contact with reality.
What makes the loop different from a chatbot is not intelligence. It is that the model's output now feeds back into its own next input, through the world. A chat model's mistake produces a bad paragraph. An agent's mistake produces a bad observation, which produces a worse decision, which produces a worse observation. The system has no external reference telling it the trajectory has gone wrong.
The Arithmetic Nobody Puts on the Slide
Take a per-step success rate. Suppose an agent gets each individual step right 95 percent of the time, which is generous for anything non-trivial.
A five-step task completes cleanly about 77 percent of the time. A twenty-step task, about 36 percent. A fifty-step task, about 8 percent.
This is the central engineering fact of agent systems and it explains nearly every pattern you see in ones that work. It is why successful deployments are short-horizon. It is why recovery matters more than raw capability — an agent that detects and corrects its own errors breaks the exponent, and one that does not is bounded by it. It is why verification is where the effort goes.
It also explains the demo-to-production gap. Demos are short, chosen, and run in clean environments. Your work is long, arbitrary, and runs in an environment full of undocumented state.
Why Coding Agents Got There First
Software engineering is the leading application, and the reason is not that programmers are early adopters. It is that code is one of the few domains with a free, fast, honest oracle.
The agent can run the tests. It does not have to reason about whether its change worked; it can execute and find out. That closes the loop with ground truth at every step, which is precisely the thing that breaks the compounding-error exponent.
In practice this means the value you get from a coding agent correlates with your test coverage almost more than with the model. On a well-tested codebase, mistakes get caught inside the loop and corrected before you ever see them. On a poorly tested one, the agent produces a large, confident, plausible diff, and now a human has to review code written by something that does not understand the system. That review is often slower than writing it yourself, which is the hidden cost nobody prices in.
The same logic identifies where else agents work: anywhere there is a cheap automatic check. Data pipelines with schema validation. Configuration with a linter. Infrastructure with a dry-run mode. Where the check is expensive, slow, or subjective, the loop cannot close and the agent is guessing.
The Components, Honestly Assessed
The model does the reasoning, and it is rarely the binding constraint any more. Swapping a good model for a slightly better one usually changes agent performance less than fixing the tool definitions does.
Tools are where most of the real engineering lives. An agent is only as capable as the actions available to it, and tool design is genuinely hard: the description has to be unambiguous enough that the model picks the right one, the arguments have to be validated because the model will produce malformed ones, and errors have to come back as messages the model can actually act on. A tool that returns a stack trace teaches the agent nothing. A tool that returns "the date field must be ISO 8601, you sent 03/04/2026" gets corrected on the next iteration. I have gained more from rewriting error messages for a model audience than from most model upgrades.
Memory is oversold. The elaborate architectures — episodic, semantic, vector-backed, hierarchical — mostly reduce in practice to deciding what goes into the context window on this iteration. Long context windows solved much of this by brute force. What remains genuinely hard is knowing what to forget, because an agent that carries every failed attempt forward spends its reasoning re-litigating history.
Planning works better when it is shallow. Agents that produce an elaborate upfront plan tend to follow it past the point where it stopped making sense. Agents that plan one or two steps ahead and re-plan after each observation adapt better. The elaborate plan looks more intelligent in a transcript and performs worse.
Guardrails are not a safety feature bolted on at the end; they are the reason the system is deployable at all. Scoped credentials, an explicit allowlist of actions, approval gates on anything irreversible, spending limits, and a complete action log. If you cannot reconstruct exactly what an agent did and why, you cannot operate it in anything that matters.
Multi-Agent Systems, and Why I Am Sceptical
The fashionable architecture is a team of specialised agents — a researcher, a writer, an editor, a critic — passing work between them.
It is appealing and it is usually worse. Every handoff between agents is a lossy communication channel, because the entire state has to be serialised into text and re-interpreted by a model that lacks the originating context. You have taken a system with compounding errors and added translation loss at each boundary. The failures also become much harder to debug: when a five-agent pipeline produces something wrong, locating where it went wrong is genuinely difficult.
There is a real case for splitting agents: when subtasks need different tools, different permission levels, or genuinely parallel execution. Separating an agent that can only read from one that can write is a sound security boundary. Splitting by job title because it mirrors how a human team is organised is anthropomorphism, not architecture.
My default is one agent with a good tool set, and I split only when there is a specific reason I can articulate.
Security Is the Underrated Problem
Prompt injection is not a theoretical concern for agents; it is the central one, and it does not have a clean solution.
An agent that reads web pages, emails, documents or issue trackers is consuming text written by people who are not its operator. Some of that text can contain instructions. The model has no reliable way to distinguish data it is supposed to process from instructions it is supposed to follow, because at the representation level they are the same thing. An agent with both the ability to read untrusted content and the ability to take consequential actions is a system where an attacker who controls the content can influence the actions.
The mitigations are architectural rather than clever. Do not give the same agent broad read access to untrusted sources and write access to sensitive systems. Treat retrieved content as data with explicit boundaries. Require human confirmation for outbound communication and anything irreversible. Log everything. Assume the model will be fooled some of the time and design so that being fooled is survivable.
What I Would Tell Someone Starting
Pick a task where mistakes are cheap and verification is automatic. Write the check before you write the agent — if you cannot state programmatically what "done correctly" means, you are not ready to automate it.
Build the evaluation set out of real tasks from your own work, fifteen or twenty of them, and run it repeatedly. Agent behaviour is stochastic; a single successful run tells you almost nothing. The same task run ten times will show you the failure distribution, which is what you actually need to know.
Instrument every action before you deploy anything. When an agent does something surprising — and it will — the log is the only thing standing between you and guesswork.
Start with approval gates on everything and remove them one at a time as you build evidence. It is far easier to loosen a constraint than to explain why you did not have one.
Where This Actually Goes
The trajectory I find plausible is that agents keep expanding into domains with cheap verification, and stall where verification is expensive. That is not a small territory, and coding is only the first of it.
What I would not bet on is autonomous operation over long horizons in messy environments, which is what most of the marketing depicts. The compounding-error problem is structural rather than a matter of one more model generation, and every real fix — checkpoints, verification, human confirmation — involves putting people back in the loop at intervals.
Which is fine. Delegation with review is how competent organisations already work. It is a genuine multiplier when done well. It is just a much less exciting sentence than "autonomous AI workforce", and the gap between those two sentences is where most of the disappointment of the last two years has come from.
Tags
Taresh Sharan
support@sharaninitiatives.com