An agent is a loop. Everything hard lives around it.

Three words get used as if everyone already agrees on them: the agent loop, the harness, and memory. This piece builds all three from nothing, one problem at a time, using what Anthropic, OpenAI, Microsoft, Google and the research papers have actually published about how they do it.

I started from a good article by Richmond Alake at Oracle, which describes the agent loop in three levels. It's worth reading. But I kept finding that each idea only made sense once you'd hit the problem it solves. So this is written as a chain. Each fix creates the next problem, and by the end you have the whole thing: the loop, the code around it, and how an agent remembers.

No prior knowledge assumed. Every term gets a plain explanation the first time it appears.

A model can't do anything, so we put it in a loop

Start with what a language model is. Text goes in, text comes out. That's the whole of it.

It can't read a file, search the web or send an email. When an AI product "searched the web", your code did the searching. The model wrote a request, something like search for X. Code outside the model ran it and pasted the result back in as more text. The model read that and decided what to ask for next.

Do that repeatedly and you have an agent. Anthropic's definition is plain:

They are typically just LLMs using tools based on environmental feedback in a loop.

The tools are the actions your code is willing to run on the model's behalf. The loop is your code going round: give the model the context, let it decide, carry out what it asked for, add the result, and go again. Anthropic's Agent SDK describes the cycle in four words:

gather context -> take action -> verify work -> repeat

Coding assistants are the clearest example. The model reads the request, looks at the code, edits a file, runs the tests, sees the failures, and edits again until they pass. Nobody wrote those steps in advance. The loop found them.

An agent is a model in a loop that your code runs. The model only ever decides; your code does.

A loop needs a way out, and "the model stopped" isn't one

Every loop needs a condition that ends it. The obvious one is built into the APIs: when the model replies without asking for any tool, the turn is over. The Claude Agent SDK documentation says it directly:

Turns continue until Claude produces output with no tool calls, at which point the loop ends.

That's where the first misunderstanding lives. Alake's article puts it well:

a terminal message from the model, one with no further tool calls, ends the agent's turn. It does not mean the user's goal has been satisfied.

The model might have stopped to ask a question. It might have given up. It might be confidently wrong. So a well-built loop has several exits, and only one of them is "the model went quiet":

A goal check — a test that the job is actually done, such as the tests passing.
A turn limit — a maximum number of rounds.
A budget — the SDK docs call this "a good default for production agents".
A timeout — a wall-clock limit.
A stuck detector — the same tool called with the same arguments three times in a row is, in Alake's words, "a strong signal that it is stuck."

Deciding when the agent is finished is your code's job, not the model's.

Every round re-sends everything, and that gets expensive fast

Here's the part that surprises people. A model has no memory between calls. So on every round of the loop, your code sends the whole conversation again: the instructions, the request, every tool call so far and every result.

Round one sends a little. Round ten sends everything from rounds one to nine, plus the new bit. OpenAI's engineer Michael Bolin, writing about the Codex loop, asks the obvious question and answers it:

isn't the agent loop quadratic in terms of the amount of JSON sent…? And you would be right.

In plain terms: the total text sent grows much faster than the number of rounds. Anthropic measured the effect on its own systems. Agents "typically use about 4× more tokens than chat interactions, and multi-agent systems use about 15× more tokens than chats". A token is the unit models read and bill by, roughly three-quarters of a word.

The saving grace is prompt caching. If the start of a request is identical to a recent one, the provider can reuse the work it already did. Claude prices a cache read at 0.1 times the normal input price. OpenAI says cached input is "discounted up to 95%". And Bolin explains why Codex is built the way it is:

the old prompt is an exact prefix of the new prompt. This is intentional.

So the rule for the loop is: only ever add to the end. Rewrite an earlier message to tidy it up, or change the tool list mid-conversation, and the cache breaks. Bolin warns that changing tools mid-run "can cause an expensive cache miss."

The loop is cheap only if the start of every request stays exactly the same.

A fuller window doesn't just cost more; the model gets worse

Cost isn't the only problem with a growing conversation. The model's context window is the amount of text it can hold at once. Fill it up and quality drops, well before you hit the limit.

Anthropic calls this an "attention budget":

as the number of tokens in the context window increases, the model's ability to accurately recall information from that context decreases.

Chroma's researchers tested 18 models and found the same thing: "their performance grows increasingly unreliable as input length grows." Their name for it is context rot. A focused prompt of a few hundred tokens beat the same question buried in a full prompt of over 100,000.

And position matters. A 2023 study, Lost in the Middle, found that "performance is often highest when relevant information occurs at the beginning or end" of the input, and "significantly degrades" when it's in the middle. A long agent run puts most of its history exactly there.

So the loop has two reasons to keep the window small: money, and accuracy.

A long conversation is the model working with a cluttered desk.

The code around the model is called the harness

Everything so far — running the loop, deciding when to stop, keeping the window lean — is done by code that isn't the model. That code has a name. Microsoft's Agent Framework documentation defines the harness as:

the runtime scaffolding that turns a language model into an agent… It drives model and tool calls, manages conversation state and context, applies approval policies.

Alake makes the same split: the model is one layer, the harness is the other, and "most agent engineering work happens in the harness, not the model."

The harness's main job is managing the window. Three techniques do most of the work.

Compaction — when the conversation gets long, summarise it and start a fresh window with the summary. Anthropic warns about the cost: "overly aggressive compaction can result in the loss of subtle but critical context whose importance only becomes apparent later." Microsoft's version has a detail worth knowing: a tool call and its result must be removed together, because removing one without the other "would cause LLM API errors."

Offloading tool output — a single web search can return "three to four thousand tokens of raw results", in Alake's numbers. Keep the full result in storage and put a short reference in the window instead. Anthropic describes agents that "maintain lightweight identifiers (file paths, stored queries, web links, etc.)" and load the detail only when needed.

Finding tools instead of listing them — every tool's description sits in the window on every round. Anthropic found "58 tools consuming approximately 55K tokens before the conversation even starts." Their fix searches for the right tools when needed. It cut tokens by 85% and raised one model's accuracy at picking tools from 49% to 74%.

There's one more harness job that matters the moment an agent does anything real. If a tool call fails and the harness simply retries it, it can run a payment or send an email twice. The fix, from Alake again, is "assigning each tool call a stable key" so a retry is recognised as a retry. That's called idempotency, and it's the harness's job, not the model's.

The model decides. The harness decides what the model gets to see.

When the window is wiped, memory is what's left

Compaction throws detail away, and a new session starts with an empty window. If an agent should know anything tomorrow that it learned today, it has to be written down somewhere outside the window. That's memory.

The clearest map comes from a 2023 research paper called CoALA. It describes "short-term working memory and several long-term memories: episodic, semantic, and procedural." In plain English:

Working memory — what's in the window right now.
Episodic memory — what happened before. Past conversations and runs. The paper: "Episodic memory stores experience from earlier decision cycles."
Semantic memory — what's true. Facts about the world, the user, the organisation.
Procedural memory — how to do things. The instructions and code the agent runs on.

The platforms are converging on the same shapes under different names. Microsoft's Foundry Agent Service memory (in preview) keeps "User profile memory", "Chat summary memory" and "Procedural memory". LangChain separates thread-scoped memory, which lasts one conversation, from long-term memory saved in named "namespaces". Google's ADK distinguishes the session, the state within it, and "Searchable, Cross-Session Information".

An older paper, MemGPT, gave the best mental model: treat the window like a computer's fast memory and the stores like its disk, moving data between "fast and slow memory" as needed. The harness is the operating system doing the moving.

Memory isn't the model learning. It's the harness writing notes and reading them back.

When memory is read and written is a design decision

Once memory exists, the harness has to decide when to touch it. There are two options, and both have a cost.

Automatic — the harness always does it. OpenAI's Agents SDK loads the conversation history "Before each run" and stores the new items "After each run", with no decision by the model. Alake's rule of thumb: "The agent should never have to decide whether to load its own conversation history."

Agent-triggered — the model decides. It searches its notes, expands a summary, or writes something down when it judges it useful. Google's ADK loads memory "when your agent decides it would be helpful."

Get the split wrong and you either fill the window with things nobody needed or miss things that should always have been there. Timing matters too. LangChain points out that saving memory "on the hot path" — during the conversation — slows the reply, so some systems write in the background afterwards.

Anthropic's memory tool takes a blunt approach to all of this. Its suggested instructions to the model begin:

ASSUME INTERRUPTION: Your context window might be reset at any moment.

Combined with clearing old tool results, Anthropic reported its agents doing 39% better on its internal tests, using 84% fewer tokens on one long task. Those are Anthropic's own numbers on its own tests, but the direction is the point: a smaller window plus good notes beat a bigger window.

Load what's always needed automatically. Let the agent fetch the rest.

Memory is also where agents get fooled and go stale

Anything an agent writes to memory, it will later read back and trust. That makes memory a target.

Microsoft's AI Red Team tested this on an email assistant. A single planted email told it to remember to forward sensitive messages to an outside address. That worked four times in ten. Then the team made the assistant check its memory more diligently before answering:

After modifying the agent's system prompt to encourage the agent to check its memory before responding, the attack success rose to over 80%.

The feature meant to make it more helpful made it easier to fool. This isn't only a lab result. In February 2026 Microsoft reported finding 50 real examples, from 31 companies, of web content written to plant instructions in AI assistants' memories.

Memory also goes out of date. A fact stored in March can be wrong by June, and the agent will repeat it with full confidence. And memory holds personal data, so someone has to be able to find and delete it. Foundry's memory offers a default expiry and per-item delete for exactly this reason. The CoALA paper flags one more risk: changing an agent's procedural memory, its own instructions, "is significantly riskier" than adding facts.

Every memory write is a decision about what the agent will believe tomorrow.

Good harnesses get smaller as the models improve

The last lesson comes from the teams that build the biggest harnesses. Anthropic's engineers write that harnesses "encode assumptions about what Claude can't do on its own… those assumptions… can go stale as models improve." One of their own techniques, resetting the context during long runs, eventually became, in their words, "dead weight".

For work that spans many sessions, their answer isn't a cleverer summary. It's plain notes the agent keeps for itself: a progress file and a git history it can read at the start of each session, because "compaction isn't sufficient."

That's a useful test for any agent design. If the harness is fighting the model, check whether the model still needs the help.

The trap

The tempting conclusion is that a better agent needs more: a bigger context window, more memory, more tools.

Each of those is more text in the window, competing for the model's attention. Each one costs more on every round of the loop. And memory in particular is one more place where something the agent read can turn into something the agent believes.

The best systems described here do the opposite. They keep the window small, keep the start of every request identical, load only what's needed, write memory deliberately, and treat everything the agent reads as untrusted until checked.

So when you look at an agent, don't ask how much it can hold. Ask what its harness keeps out.