Content Understanding — analyzers and field methods

Card 24 covered Content Understanding for images and video. This one covers what it does with documents, and the detail here is finer — particularly the three field methods, which decide most of the questions.

The analyzer workflow

1. Choose a base analyzer — prebuilt, or blank.
2. Define the field schema — name, type, description, method.
3. Select the model.
4. Build — this compiles into a reusable analyzer ID.
5. Analyze — begin_analyze against your files.

The three field methods

Method	What it does	Its constraint
extract	Pulls the value verbatim out of the source	Documents only, and needs grounding switched on
classify	Assigns a predefined category from an enum	Also used for routing
generate	Freely produces a value — summaries, computed totals	Not verbatim

The distinction between extract and generate is the one that matters. Extract finds a value that is literally present. Generate produces one that is not — a summary, or a total nobody wrote down.

Confidence and results

This is the part with the most practical weight, and it is worth reading carefully if you care about automating anything.

• Confidence runs from 0 to 1.
• Enable it with estimateFieldSourceAndConfidence or estimateSourceAndConfidence.
• A confidence of 0.9 or above is treated as safe for automated straight-through processing. Below that, route to human review.
• Every result returns three things: the value, its location — page and bounding box — and a confidence score.

That triple is the whole basis for trusting extraction at scale. A value alone tells you what the system thinks. A value with a location and a confidence tells you where it came from and how sure it is, which is what lets you decide between processing it automatically and putting it in front of a person.

Prebuilt analyzers

There are prebuilt analyzers for invoices, receipts, identity documents, tax forms and contracts. If one exists for your document type, use it — hand-building a schema for receipts when a receipt analyzer exists is effort spent recreating something better tested than your version.

Output and reach

• It emits Markdown suited to retrieval, preserving tables, figures and layout.
• One analyzer handles documents, images, audio and video.
• Audio and video additionally get transcription, diarization and frame analysis.
• Prefer Entra ID authentication; schemas export as reusable JSON.

Content Understanding or Document Intelligence

Content Understanding is generative. The schema is written in natural language, it covers all modalities, and it returns confidence and Markdown.

Document Intelligence uses layout-specific machine learning models. Its Read capability handles optical character recognition and text; its Layout capability handles tables, structure and selection marks.

Reach for Document Intelligence when you need precise structural analysis of a document's layout. Reach for Content Understanding when you want fields you have described in your own words.

The trap

extract is documents-only.

So a scenario asking you to pull a value verbatim out of a video frame or an image cannot use it — and that makes generate the right answer there, even though the word "verbatim" in the requirement points hard at extract.

This is the same shape as the Pro mode trap on card 24: everything in the scenario matches the obvious answer except the media type, and the media type decides it.