Token and cost engineering

In a single-agent system, tokens are roughly: instructions, plus retrieved context,
plus the conversation, plus the output. Multi-agent adds three multipliers that
people consistently underestimate.

Where the cost actually comes from

Every hop re-sends context. A five-agent chain that passes the conversation
along sends overlapping context five times. This is the strongest cost argument for
passing a deliberate payload instead.

Every loop re-sends everything. An agent that goes round four times pays for the
accumulated context four times — and the fourth pass is the most expensive, because
context has grown.

Every orchestrator turn is a model call. A Magentic-style manager reasons before
each delegation. That is a full call, on top of whatever the subagents do.

Put together: in single-agent systems you pay for what the user said and what you
retrieved. In multi-agent systems you mostly pay for agents talking to each other
about the work — and that portion grows faster than the work does.

The ladder, in order

Work it top-down. The order matters because the cheap wins are at the top and the
expensive, slow ones are at the bottom, and teams tend to start at the bottom.

1. Trim what tools return. Six fields instead of the whole record. This is
usually the single cheapest optimisation available: it cuts tokens on every
subsequent call in the run, it improves answer quality by removing noise, and it
takes an afternoon.

2. Trim the prompt.

3. Fix the handoffs — payloads, not conversations.

4. Right-size the model per step. Per step, not per system. The step that
classifies an intent does not need the model that writes the final answer.

5. Cache. Prompt caching first — nearly free and no correctness risk. Semantic
caching saves more and carries real risk.

6. Reduce agents. Does this step actually need its own agent?

7. Fine-tune a smaller model. The slowest and last resort.

Loop controls: you want all three

Max iterations bounds runaway reasoning within one agent.

Max depth bounds recursive subagent spawning.

A budget per request — in tokens or cost — bounds everything else.

The third is the important one, and the reason is worth understanding: iteration
and depth limits can each be respected while the combination is ruinous. Eight
subagents, each looping nine times, is within a limit of ten iterations and a depth
of two. It is also seventy-two agent runs.

The budget is the only control that sees the whole picture.

Enforce it in the orchestrator, not the prompt — "try to be efficient" is not a
control — and make it fail loudly. A silently truncated run produces a confident
partial answer, which is worse than an error, because nobody knows to distrust it.

(That the budget belongs in the orchestrator is an engineering position rather than
a documented Microsoft requirement — though the next card's evidence about how
slowly Azure budgets evaluate makes the case fairly hard to argue with.)

Tool calls as a cost line

Easy to forget because each one looks cheap.

Every result re-enters context and is carried for the rest of the run. A tool
returning a large JSON blob costs you on every subsequent call, not once.

Failed calls cost full price. Those four retries against a permanent failure
were four paid round trips.

Chatty tools invite chatty use. A tool that returns everything encourages the
agent to call it repeatedly rather than precisely.

The trap

Parallelism cuts wall-clock time, not tokens.

And "we parallelised it and it did not get faster" is a rate limit question, not a
concurrency one. Fan out eight ways against a deployment that permits four
concurrent calls and four of them simply queue — you have added complexity and
achieved nothing.