Rotation, certificates and encryption

Key Vault holds three kinds of thing, and the distinction between two of them is
worth more than it first appears.

Secrets, keys, certificates

Secrets are arbitrary strings — API keys, connection strings, passwords. You
retrieve them, which means they travel to wherever your code is running.

Keys are cryptographic keys used for operations rather than retrieved. You ask
the vault to perform an operation with the key. The key never leaves.

Certificates come with lifecycle management including renewal. Where the
counterparty supports certificate authentication, prefer them over a secret — a
certificate has an expiry the system understands and can act on, rather than a
password that is valid until someone remembers to change it.

The plain-English version: a secret is something you take out and use. A key is
something you send work to. Anything you can express as the second is safer,
because it never travels and therefore cannot be intercepted, logged, or left in a
variable.

Rotation, and why agents are unusually bad at it

Rotation is the obligation that makes secrets expensive. Three rules:

Rotation must not require a deployment. If changing a credential means shipping
code, it will not happen on schedule. Read from the vault at run time, with a cache
short enough that rotation takes effect promptly.

Handle the overlap. During rotation both the old and new credentials are briefly
valid. Code that caches indefinitely survives the rotation and then fails after the
old one is revoked — usually hours later, and confusingly, because nothing changed
at the moment things broke.

Rotate on compromise, not only on schedule. And know how long a rotation takes
end to end, because that number is your exposure window when something leaks.

The agent-specific wrinkle: long-running agents are unusually prone to the caching
failure, because they are long-lived processes. A credential fetched at process
start and held in memory survives every rotation until the process restarts — and
agent processes are designed not to restart.

Certificates rotate too, and they expire whether or not you were ready. An
expired certificate is an outage with a date you could have known about months in
advance.

Encryption

At rest. Key Vault encrypts its contents. The design question is whether you
need customer-managed keys, where you hold the key rather than Microsoft. For
regulated deployments that is often a requirement rather than an option — and it is
a control with a real operational cost, because you now own key availability and
losing the key means losing the data. Take it where a regulator or a contract
requires it, and be honest that the obligation is real.

In transit. TLS is a given. The design question is the private-endpoint one:
should traffic to the vault traverse the public internet at all?

What must never reach a prompt

A short section because it is short to say and expensive to get wrong.

Never put a secret in an instruction block, an example, or a tool description.
Everything in a prompt can end up in a trace, an evaluation record, a training set
or a summary — and a secret that entered any of those has effectively been published
internally.

The same applies to tool results. A tool returning a full connection string into
the conversation has put a credential into context, memory and telemetry in a single
call.

The trap

The case nobody plans for: a downstream error message that helpfully includes the
connection string it failed on.

Nobody wrote that. The library did. And it lands in the agent's context, gets
carried through the rest of the run, appears in the trace, and possibly ends up in
an evaluation record.

The defence is a small piece of middleware that scans tool outputs for
credential-shaped strings before they enter context — which is unglamorous, cheap,
and catches the failure that no amount of careful prompt authoring would have
prevented.