Choosing a model — the four leaderboard metrics

There are a lot of models to choose from, and the platform gives you a scoreboard to compare them. The scoreboard is only useful if you know what each column is actually measuring — and two of them get confused constantly.

The six columns, and which way is good

What it is called	What it actually measures	Which direction is good
Throughput	How quickly prompts are processed and answers come back	Higher
Fluency	Whether the language is well formed — grammar and phrasing	Higher
Groundedness	Whether the answer reflects your real source data	Higher
Relevance	Whether the answer addresses the question that was asked	Higher
Cost	US dollars per million tokens, priced separately for input and output	Lower
Safety	Attack success rate — how often an attempt to misuse the model works	Lower

Read the last column carefully. Safety is the only metric where a lower number is better. Every other one rewards a higher score. That asymmetry is the single easiest thing to get wrong under time pressure.

Fluency is not groundedness

This is the distinction the exam keeps returning to, and it matters well beyond the exam.

Fluency is about form. Does it read like competent English? A model can be perfectly fluent and completely wrong. Fluent nonsense is still nonsense — it is just harder to spot.

Groundedness is about truth. Is the answer anchored in the source material you gave it? This is the closest thing the scoreboard has to a hallucination detector, where a hallucination is the model stating something confidently that its sources do not support.

If you only watch one of these in production, watch groundedness. Fluency is nearly always fine; that is what these models are good at.

The token maths behind the numbers

A token is a chunk of text the model reads and writes in. Roughly, one token is four to eight characters — not quite a word, a bit more than a syllable.

Two things follow that are easy to miss:

• The leaderboard's cost figures assume a three-to-one ratio of input to output. Three tokens in for every one out. If your real workload is shaped differently — long documents in, short answers out, or a short question producing a long report — the published cost will not match your bill.
• The context window is shared between the prompt and the response. It is not a budget for your input with the answer on top. A long prompt leaves less room for the answer.

Testing a model automatically

You do not have to judge outputs by hand. Automatic evaluation uses one model to score another's answers — an AI judge — run across a set of prompts. If you do not have a set of test prompts, the wizard can generate a synthetic one for you.

One piece of judgement here: when you are testing a model directly, drop the intent-resolution and business evaluators. Those are designed for multi-turn agents — conversations with several back-and-forth steps — and they will not tell you anything useful about a single model call.

Two ways you get billed

Worth knowing because it changes procurement, not just cost.

• Azure-sold models go through Azure meters, on your existing subscription.
• Partner and community models are billed through the Azure Marketplace instead. Different paperwork, and often a different approval path inside a large organisation.
• Instant models, in preview at the time of writing, let you call a model by name with no deployment step at all.