Card 14 of 40· Develop

Tools: integration, failure and validation

Four kinds of tool failure with four different answers, and why a tool result is untrusted input arriving after your input filter has run.

Tools: integration, failure and validation
Open the card in a new tab to read it at full size.

Tools are how an agent does anything at all, and the syllabus asks for four things:
integration, error handling, fallback, and result validation. The last two are
where most implementations are thin.

Three ways a tool reaches the agent

Function calling. You define the schema; the model decides when to call it.

Specified tool use. You constrain which tool may be used, and when.

Dynamic tool use. The available set changes at run time.

That third one deserves a moment, because it is easy to read as a convenience
feature. The set of tools you expose is a permission decision, made per turn.
Deciding that an agent gets the refund tool only after identity has been verified is
not a UX choice — it is access control implemented in the tool layer, and it is
often the cleanest place to put it.

Four kinds of failure, four different answers

Treating all tool failures the same is the most common mistake here, and it is
expensive in both directions.

Transient — a timeout, a 503, a rate limit. Retry with backoff, bounded.

Permanent — a 404, a 403, a schema mismatch. Do not retry. Four retries
against a permanent failure are four paid round trips producing nothing, and they
add latency to a request that was always going to fail.

Degraded — it answered, but partially or slowly. You have to decide in advance
whether partial is usable, because the agent will use it either way.

Wrong — it answered confidently and incorrectly. No retry policy helps. Only
validation catches this one.

Fallback

Fallback is the plan for when a tool is unavailable, and it has to be a designed
plan rather than an emergent one.

The failure mode without it: the tool fails, the agent notices it cannot get the
answer, and improvises. It answers from what it already has in context, or from
what it knows generally, and presents the result with exactly the same confidence
as it would have had with real data.

So the useful fallback options are: an alternative source, a cached or stale answer
labelled as stale, a graceful degradation to a narrower answer, or an escalation.
What they have in common is that the user can tell the difference.

Result validation

The step most teams skip, in two stages.

Schema first. Does the shape match what you declared? Deterministic, cheap, and
it catches most of it.

Then plausibility. Is the value in a possible range? A price of zero, a date in
1900, a customer record with no identifier — the shape is fine and the content is
not.

And the point that connects this card to the security ones:

A tool result is untrusted input. It arrives after your input filter has run
and passed. If an attacker can influence a document your agent retrieves, an issue
body it reads, or an API response it requests, they can put instructions into your
agent's context without ever typing anything into your interface.

Which is why the guardrail card lists tool responses as one of the four
intervention points, and why it is the one people forget.

The trap

There is a gap between what an agent intends and what it is allowed to do,
and that gap is the most valuable control point in the whole system.

An agent that has been persuaded to want something harmful is contained if the tool
simply is not available to it. The persuasion succeeded and the damage did not.

Which is the same conclusion the specification card reaches from a different
direction — and it is the reason "restrict the tools" beats "improve the
instructions" essentially every time it is offered as an option.