◆ The signature piece
The AI Engineering Pipeline
The model is the easy part. The hard part — and the whole value — is the reliable system you build around a model that is, by nature, unpredictable. And the trend line is clear: with every passing month the leverage keeps moving out of the model and into the system around it. This is that system, mapped end to end: the lifecycle, the frameworks, the SDKs, and the way I carve the craft into Prompt, Context and Loop engineering.
A living draft · written from the workbench, not the whitepaper · current as of mid-2026 · updated as the field moves
◆ The thesis
The model is the easy part.
Anyone can call a model now. It is one line of code and a key. The model is a commodity, and it gets better while you sleep — for free. So the model is not where the value is.
The value is everywhere else: framing the real problem, feeding the model the right context, checking its work, catching its mistakes, and keeping it honest in production. That surrounding machinery is the discipline. Building it well is what I mean by AI engineering.
One line runs through this whole page. The same model that writes a flawless 200-line function can lose the plot halfway through a 2,000-line feature — and the difference is not the model, it is the scaffolding around it. Anthropic calls that scaffolding the harness. Skills, memory, retrieval, guardrails, evals: they are all just leverage that used to live in the prompt and now lives in the system. That migration is the story.
◆ The pipeline
One idea, six stages, end to end.
Every AI system I build walks the same path. The tools change; the shape does not. Two things are worth saying out loud before the diagram: most of the real work is at the front, in framing the problem and designing for it — not in the code. And most of the real risk is at the back, in monitoring — because models drift and the inputs people send you shift under your feet.
- 01IdeaA hunch that a model could help. Cheap to have, easy to over-trust.
- 02ProblemFrame the real job to be done, and what "good" measurably looks like.
- 03DesignDecide the data, the context, the tools, and the guardrails up front.
- 04BuildWire the model, prompts, retrieval and tools into a working system.
- 05DeployShip it behind auth, limits and logging — the boring parts that matter.
- 06MonitorWatch quality, cost and drift. Evals here decide if it lives or dies.
Beginners obsess over the middle — the build. Experienced teams spend their attention on the two ends. A sharp problem statement kills bad ideas cheaply, and good monitoring is the difference between a demo that impressed a room once and a system you can trust on a Monday morning.
Where systems live or die: the reliability stack.
The last stage — monitor — is not one thing. It is three layers, and mature teams run all of them. Skip a layer and the failure it was meant to catch is the one that reaches your users. This is the least glamorous part of the field, and the part that actually decides whether a system survives contact with the real world.
Offline evals
A fixed set of test cases you score every change against, in your build pipeline, before anything ships (these are your evals). Treat them like unit tests: every new prompt, model or pipeline tweak has to pass the suite before it merges. This is what stops a “small” prompt edit from quietly breaking last month’s fixes.
Guardrails
Checks that sit in the live request path and inspect each input and output — block it, rewrite it, or escalate to a human (these are guardrails). Evals judge the system in the lab; guardrails police it in the moment, on the one weird request your test set never imagined. They are the seatbelt, not the crash-test.
Observability & drift
Record every step of every real run — prompts, tool calls, tokens, latency — so you can debug behaviour that is never the same twice (a saved run is a trace; the tooling has largely standardised on OpenTelemetry). Then sample live traffic and keep scoring it, so you catch quality quietly drifting — the change that happens to you, not one you made.
Here is the honest heart of it: the evals are the moat. Not the model, not the framework — the disciplined way you measure quality. And good evals do not start with clever tooling. They start with looking at your own data — reading real failures, sorting them by cause, and counting how often each one happens. When you use a model to grade another model (“LLM-as-judge”), keep the verdict binary — pass or fail, aligned to what one real domain expert would say. Hamel Husain, whose evals guidance the field leans on, is blunt about the fancier version: if your eval is a pile of 1-to-5 scores an AI made up, you are doing it wrong — nobody knows what to do with a 3 versus a 4. Simple, honest, and never stop looking at the data.
◆ The framework landscape
Reach for a framework to remove real glue code — not by default.
A framework is pre-built plumbing: someone else wrote the code that connects your prompts, your model, your tools and your memory so you do not have to. That is useful when it saves you real work, and a liability when it hides what is actually happening. The landscape shifted through 2025–2026, so here is the current read — figures below are directional, and version strings churn month to month.
| Option | What it is | Reach for it when |
|---|---|---|
| LangChain 1.0 | A toolkit that wires prompts, models, tools and memory together with ready-made building blocks. Since the joint 1.0 release (Oct 2025) it runs on the LangGraph runtime, and its create_agent is the fast path to a working agent. | You want to ship fast and value provider-agnostic integrations. The cost is control: when it breaks, the stack trace surfaces from deep inside framework code, not yours. |
| LangGraph | Models an agent as a graph — explicit steps and shared state you can see, loop over, pause for a human, and resume after a crash. It is the low-level engine underneath LangChain, from the same team. | You need cycles (re-plan / retry / reflect), durable state across turns, or human-approval gates. Overkill for a simple linear task; the payoff is what survives production. |
| LlamaIndex | A toolkit built first for feeding your own documents to a model — it ingests messy files (tables, images, layouts), indexes them, and fetches the right chunks fast. | The hardest part of the app is messy, document-heavy retrieval and you want sensible defaults. The precision leader on ingestion; narrower on general multi-step orchestration. |
| DSPy | Instead of hand-tuning prompt wording, you declare the inputs and desired outputs, give it graded examples, and let it compile the best prompt. Prompts become tuneable parameters, not static strings. | You have a measurable task and evaluation data — extraction, classification, multi-hop QA. Without eval data or on open-ended creative work, the machinery buys you little. |
| Just the SDK | No framework — the model maker’s own library plus orchestration you write yourself. Anthropic’s own advice is to start here: many agent patterns are a few lines (a while-loop over model→tool→model). | Most production systems, honestly. Full control, nothing hidden, no version churn you did not sign up for — at the cost of owning the plumbing yourself. |
◆ The SDKs & the standard underneath
A thin, typed wrapper over an HTTP call.
Under every framework is something simpler: the official library the model maker ships — Anthropic, OpenAI, Google each have one (these are the SDKs, software development kits). Strip away the marketing and an SDK is a small, typed wrapper around one web request to the model. You send messages; you get a reply.
What they give you is the useful stuff on top of that request: a clean messages format, tool-calling (letting the model ask your code to run something), streaming (words arriving as they are generated), and structured output (replies forced into a fixed shape you can parse). Most real systems are “SDK plus my own orchestration” — not a big framework.
The Agent SDKs
Within days of each other in 2025, OpenAI, Google (ADK) and Anthropic all shipped official Agent SDKs — a middle layer between raw calls and a heavy framework. Same realisation: raw calls need too much plumbing, full frameworks are too opinionated. So they ship the useful primitives — tools, handoffs, sessions, memory, MCP — and leave the opinions to you. They differ in taste: Google’s ADK treats agents as software components in a distributed system, OpenAI’s as conversation handlers with delegation, and the Claude Agent SDK as computer users with tool access, skills and context compaction built in.
MCP — the universal socket
The reason those SDKs interoperate is one shared standard: the Model Context Protocol (MCP) — a common way for any model to discover and call your tools and data, so you wire a tool once instead of once per vendor. Anthropic started it in late 2024; OpenAI, Google and Microsoft adopted it; and in December 2025 Anthropic donated it to the Agentic AI Foundation under the Linux Foundation, so it stopped being one company’s protocol and became neutral infrastructure. It is the biggest standardisation event of the period — the USB-C moment for agents.
◆ The centerpiece
Prompt, Context, Loop — three disciplines that build on each other.
This is the framing I keep coming back to, and the one I teach first. The whole field splits cleanly into three crafts. They stack: you cannot skip a level. Get them in the wrong order and you tune prompts while the loop is what is broken. And notice the trend as you read down — each level moves more of the leverage out of a single clever sentence and into the system around the model.
Prompt Engineering
The model only ever sees the text you hand it, in one shot. So how you write it — the instructions, the examples, the format you ask for — is the whole game at this level. Three things that still matter in 2026: be explicit and literal (vague asks get vague results); examples beat instructions (two to five diverse ones teach more than a paragraph of rules); and structured output is now an API feature, not a prompt trick you beg for. What is dated: manually writing “think step by step.” Frontier models have native reasoning built in — you turn a dial for how hard it thinks, you no longer prompt it to think. Heavy role-play personas and XML scaffolding have faded too.
Context Engineering
Everything the model reads on a call is the context, and it is a finite budget with diminishing returns — more tokens is not more quality. There is a measured failure mode here: context rot. Chroma tested 18 models and found accuracy drifts down as the input grows, even on trivial tasks — so a bigger window is not a fix. The craft is fitting the smallest set of high-signal tokens: pull facts only when needed (just-in-time retrieval — hold a file path, load it at the last moment), summarise old history (compaction), keep notes in a file outside the window (memory), and split work across sub-agents so each keeps a clean context and returns only a distilled summary. As systems scale, this is the skill that matters most.
Loop Engineering
Once the model can call tools, it stops being a one-shot reply and becomes an agent working in a cycle: plan, act, observe, repeat until a stopping rule fires. First a warning worth heeding: most jobs do not need a fully autonomous agent — a fixed workflow (predefined steps) is more predictable, and both Anthropic and OpenAI say start there. When you do build the loop, you are doing harness engineering: the retries, the budgets, the stopping conditions, the reusable skills an agent loads on demand, and the trick of letting it write code to use tools (via MCP) instead of chattily calling them one at a time. Name the failure modes and you can design against them: trying to one-shot a big task, declaring “done” without testing, and buggy handoffs between steps.
One honest flag: the loudest framing in the field — “autonomous multi-agent systems” — is oversold. The primary sources from the labs that build these things all say the same unglamorous thing: start with a single well-prompted call, prefer a workflow to an agent, and only add autonomy when a real limitation forces it. Each extra autonomous turn adds cost, latency, and the chance an early mistake compounds. Simplest thing that works, every time.
◆ How it evolved
The leverage keeps moving out of the model.
You can watch the whole field as one long trend. Each era added a layer of system around the model, and each new layer is worth more than a bigger model would have been. That is the single idea that ties this whole page together.
- Era 1
Prompt-only chat
You type, it answers. One message in, one message out. Impressive, but stateless and blind to your data. This is pure prompt engineering.
- Era 2
Retrieval & RAG
We started fetching the right documents and pasting them into the prompt so the model could answer from real, current facts. The first taste of context engineering.
- Era 3
Tool-use & agents
Give the model the ability to call your code — search, run, write files — and let it act in a loop. The birth of loop engineering, and of agents that do rather than just say.
- Era 4
The harness era
The scaffolding around the model becomes the product: durable memory, reusable skills, sub-agents that specialise, and MCP as a shared standard so tools plug in once. Compaction alone proved not enough for long runs — you need a real harness. The whole trend line, in one word: leverage moving out of the model and into the system.
◆ A working shootout
Coding agents: Claude Code vs Codex vs OpenCode.
A coding agent is a model wrapped in a harness that can read your repository, run commands, and edit files in a loop — the trichotomy made concrete. Here is my from-experience read on the three I reach for. It is a snapshot, not a verdict, and it is not winner-take-all: these have genuinely different shapes. This is exactly the kind of thing that shifts month to month, so treat the comparison as dated rather than settled.
| Dimension | Claude Code | Codex | OpenCode |
|---|---|---|---|
| Approach | Anthropic's terminal-native agent. Lives in your shell, interactive and deep; strong harness — memory, skills, sub-agents, MCP baked in. | OpenAI's cloud-first coding agent (in ChatGPT + a CLI). Async and parallel — a general computer agent that also codes. | Open-source, model-agnostic harness (terminal, desktop, IDE). It is infrastructure, not a model — you plug in whichever model you choose. |
| Openness | Closed model, but the Agent SDK is open and it is MCP-native — a scriptable, open harness with a rich skills/hooks system. | Closed model and closed stack, tied to OpenAI models and cloud; you work inside their product. | MIT-licensed, fully inspectable, forkable, self-hostable. 75+ providers — Claude, GPT, Gemini, or fully local via Ollama. |
| Autonomy | Deep interactive sessions; strongest on long, context-heavy, multi-file work with guardrails. | High async autonomy — demoed long unattended runs and many tasks in parallel; the cheap, fast grind. | As autonomous as the model you plug in — you own the ceiling and the risk; plan/build modes and an editor feedback loop. |
| Best fit | Complex codebases, architectural work, security-sensitive and context-heavy tasks where the harness earns its keep. | Cheap, fast, well-defined autonomous grind and parallel task execution; teams standardised on OpenAI. | Model choice, self-hosting, and regulated / data-residency settings where you must control the whole stack. |
The pattern under all three: same class of model, very different harness — and the harness is usually what decides the result. Reported benchmarks split by strength rather than crowning a winner (one leads on shell work, another on hard multi-file problems), and OpenCode’s runaway popularity shows how much demand there is to simply own the harness. Change the context you feed it and the loop that checks it, and you can get several times the output from the same underlying model.
◆ The part no serious engineer skips
Agentic security: the lethal trifecta.
Give a model tools and you also give it a way to be turned against you. The number-one risk is prompt injection — hidden instructions smuggled inside content the model reads (a web page, an email, a document). It sits at the top of the industry’s risk list (OWASP LLM01) and is, honestly, the hardest to fully prevent, because it does not need to be human-readable to work.
Simon Willison’s lethal trifecta
Data gets stolen when an agent has all three at once: access to private data, exposure to untrusted content, and a way to send data out. Grant all three in one session and an attacker who controls the untrusted content can read your private data and ship it out — no exploit code required. This is not theoretical: a zero-click exfiltration bug in Microsoft 365 Copilot (“EchoLeak”) was exactly this pattern. The defense is architectural — break at least one leg of the trifecta per session.
Tool misuse & the layered defense
When a model can browse, run code and call APIs, one injection’s damage scales with its permissions — individually-harmless tool calls chained into something harmful (tool misuse), or an agent’s delegated credentials used to move sideways (excessive agency). There is no single fix. You layer: least-privilege tools and scoped credentials, human-in-the-loop approval for high-consequence actions, input/output guardrails, sandboxed execution, and red-teaming in your build pipeline. The goal is simply to make the trifecta hard to assemble.
I care about this more than most because of where I work. In heavily regulated settings, the cost of an agent quietly leaking data is not an embarrassing tweet — it is a serious incident. Least-privilege, no unnecessary data egress, and a human on the high-stakes actions are not bureaucracy there. They are the whole job.
◆ Sources & further reading
Where this comes from.
I write from the workbench, but every load-bearing claim above traces to a primary source. Here are the ones I lean on — mostly the labs that build these systems, plus the practitioners who set the field’s language on evals and security. If you read only three, read the Anthropic context-engineering post, Hamel Husain on evals, and Simon Willison on the lethal trifecta.
- 01Anthropic — EngineeringBuilding Effective AgentsThe workflows-vs-agents distinction and the “start simple, avoid unnecessary frameworks” rule.
- 02Anthropic — EngineeringEffective Context Engineering for AI AgentsThe canonical piece: context as a finite budget, context rot, just-in-time retrieval, compaction, memory, sub-agents.
- 03Anthropic — EngineeringEffective Harnesses for Long-Running AgentsWhat a “harness” is, why compaction alone is not enough, and the named agent failure modes.
- 04Anthropic — EngineeringCode Execution with MCPLetting the model write code to use tools — the clearest case of leverage moving out of the model into the system.
- 05Anthropic — NewsDonating the Model Context Protocol & the Agentic AI FoundationDecember 2025: MCP becomes neutral, industry-owned infrastructure.
- 06OpenAIA Practical Guide to Building AgentsIndependent corroboration on the loop, single-vs-multi-agent, and layered guardrails.
- 07Chroma ResearchContext RotThe 18-model study that turned “long context degrades” from folklore into measured fact.
- 08Hamel HusainCreating a LLM-as-a-Judge That Drives Business ResultsThe practitioner take on evals: look at your data, binary pass/fail, align to a domain expert.
- 09Simon WillisonThe Lethal Trifecta for AI AgentsPrivate data + untrusted content + external egress — the security framing every serious engineer cites.
- 10OWASPLLM01:2025 Prompt InjectionThe authoritative risk taxonomy — #1, and the hardest to fully prevent.
- 11Dex Horthy / HumanLayer12-Factor AgentsA vendor-neutral reliability manifesto: own your context, control flow and state.
- 12Andrej KarpathyOn “context engineering” over “prompt engineering”The mid-2025 post that popularised the term the whole centerpiece is built on.
◆ This is the theory. Get hands-on.
Python for AI Engineers — a free, hands-on course.
This page maps the field. The course puts your hands on the keyboard: fifteen chapters from the Python basics to a working Streamlit → FastAPI → Gemini → SQLite capstone, written for people who can already read code but want to build with LLMs. Free and self-paced.
Start the course →◆ The teaching
Don’t chase a smarter model. Engineer a better system around the one you have.