Knowledge routing: copy or query

There are four ways an agent can reach knowledge, and they get discussed as though
they were four products. They are not — one of them is not even a destination.

The four routes

RAG is copy it, prepare it, and own the pipeline.

MCP is ask the system that owns it.

Search is use the index someone already built.

Semantic search is a property of how any of them match. It is not a separate
place to get things from.

That last one is worth being firm about, because "should we use semantic search or
RAG" is a question that cannot be answered as asked.

The decision that matters most

Almost every knowledge-integration argument reduces to one question: copy, or
query?

Copying it in — the RAG route — gives you control over chunking, filtering,
ranking and latency. Real benefits, and the reason RAG exists.

It costs you two things that are easy to underrate. A freshness problem: your
copy is only as current as your last ingest. And a permissions problem: you now
have a second access-control surface that has to agree with the original, forever.

Querying it live — the MCP route — gives you material that is current by
definition and access control that stays where it belongs. It costs you latency, a
dependency on someone else's availability, and much less control over the shape of
what comes back.

A preference this guide argues for rather than a Microsoft recommendation:
default to querying live, and copy only when you can name what copying buys —
retrieval tuning you have measured a need for, latency you have measured a problem
with, or a source that genuinely cannot serve queries.

The reflex to ingest everything into a vector store is how organisations end up
maintaining a stale second copy of data they already had, with a permissions model
that drifts from the original.

Permission trimming is the hard part

Every route has to answer the same question: does this user, through this agent,
have the right to see this? And the four routes fail differently.

Search and RAG over a copied index. Permissions have to be replicated into the
index and kept in step. This is where oversharing gets baked in — a copy made once
with permissive access control stays permissive long after the source was tightened,
and nobody re-checks.

MCP-available sources. Permissions stay with the owning system — provided the
call carries the user's identity. If the MCP server is called with a service
credential, every user of that agent gets that service's access.

That second case is worth stating as plainly as possible: a "temporary" service
credential on a knowledge connector is the single most common way an agent becomes
an entitlement bypass.

It is invisible in testing. Everything works, results look right, nobody is
harmed — because the people testing have broad access anyway. It surfaces months
later when someone asks a question they should not have been able to answer, and by
then it is load-bearing.

The connection to the toolbox argument

The copy-or-query decision is the same argument as the toolbox one in different
clothes: reuse the governed thing rather than reproducing it.

Every copy is a second thing to govern. Every reproduction is a second place for
permissions to drift. The instinct to bring data closer feels like control and is
often the opposite.

The trap

"Users must only see what they already have access to" is answered by
on-behalf-of — not by filtering results after retrieval.

Filtering afterwards means the data was already retrieved and already in the prompt.
A prompt is a place data leaks from: into traces, into evaluations, into memory,
into a summary that gets passed to another agent. The filtering answer looks careful
and arrives one step too late.