Implement information extraction solutions
Document Extraction with Content Understanding
ImportantCombine OCR, layout analysis, and field extraction to pull information from documents, use Content Understanding to produce clean markdown and grounded fields for agents and RAG, and build analyzers that return structured or markdown output for downstream reasoning.
Aligned to the AI-103 study guide with skills measured as of April 16, 2026; guide and product behavior verified October 12, 2026.
Why this matters
Invoices, contracts, forms, and reports hold the facts that agents and automations need, but raw PDFs are messy. AI-103 expects you to choose between read, layout, and field extraction, decide when Content Understanding or Document Intelligence fits, and use confidence and grounding to decide what still needs human review.
Must Know
- Content Understanding prebuilt-read performs OCR for words, paragraphs, formulas, and barcodes; prebuilt-layout adds tables, sections, figures, and structure; neither requires a language model deployment.
- prebuilt-documentSearch is the recommended analyzer for RAG ingestion: it returns markdown with paragraphs, tables, figure descriptions, and a summary, and supports chunked output.
- A custom document analyzer derives from prebuilt-document and defines a fieldSchema; fields use extract (literal values from the document), classify, or generate methods, and the service can pick the method automatically.
- Enable estimateFieldSourceAndConfidence (or per-field estimateSourceAndConfidence) to return a 0 to 1 confidence score and the source location for each field, so low-confidence values can go to human review.
- contentCategories with enableSegment splits a multi-document file into segments, classifies each, and routes it to the matching subanalyzer in one call; classification is now merged into the analyzer rather than a separate classifier.
- Analyzer output can be markdown for search and reasoning or JSON fields matching the schema; tables can be returned as HTML or markdown, and multipage tables are returned as one table.
- Document Intelligence remains the fit for high-volume, deterministic extraction from structured forms with low latency, while Content Understanding fits varied or unstructured content and inferred fields without labeling to start.
- In Azure AI Search, the Document Layout skill and the Content Understanding skill turn documents into structure-aware markdown chunks during indexing.
Compare and Distinguish
- prebuilt-read versus prebuilt-layout versus prebuilt-documentSearch: text only versus text with structure versus RAG-ready markdown with figure descriptions and summary.
- Extract versus generate fields: values copied from the document with grounding versus values the model writes, such as a summary.
- Content Understanding versus Document Intelligence: generative, multimodal, schema-first analysis versus deterministic form models with labeled training for custom models.
- Markdown output versus JSON fields: content for retrieval and reasoning versus typed values for automation.
Scenario examples
- Scenario: An accounts payable agent needs vendor name, invoice total, and due date with evidence for each value. Think: a custom document analyzer with extract fields and confidence and grounding enabled.
- Scenario: A single scanned PDF contains an application form, a pay stub, and an ID card. Think: contentCategories with segmentation that routes each part to its own analyzer.
- Scenario: A RAG index must keep table structure and chart meaning from annual reports. Think: prebuilt-documentSearch markdown with figure descriptions, chunked for indexing.
Exam traps
- prebuilt-read does not return table structure; use prebuilt-layout or a search-oriented analyzer.
- Generated fields are not literal quotes from the document; use extract when the exact value and its location matter.
- Confidence scores are returned only when estimation is enabled, and only for documents.
- Content Understanding requires your own Foundry model deployments for generative features; there is no managed capacity.
Key takeaways
- Pick read, layout, or search-oriented analyzers by how much structure the consumer needs.
- Use extract fields with confidence and grounding for automation that must be verifiable.
- Feed agents and RAG clean markdown rather than raw OCR text.
How it works
- The analyzer first extracts content and layout, then a generative model fills fields, and contextualization computes grounding and confidence.
- Analyze requests return an operation URL that the client polls until the result is ready.
Objects and administrative surfaces
- Content Understanding Studio, REST API, and SDKs (azure-ai-contentunderstanding for Python).
- Analyzer definitions with baseAnalyzerId, fieldSchema, config, and models.
- Azure AI Search Document Layout and Content Understanding skills.
- Document Intelligence prebuilt, layout, and custom models.
When to use it
- Use prebuilt domain analyzers such as invoices or tax forms before building custom schemas.
- Use a custom analyzer when your documents need fields that no prebuilt covers.
Security and governance implications
- Delete analysis results after retrieval when retention rules require it.
- Route low-confidence or high-value fields to human review before posting to business systems.
Troubleshooting signals
- If tables come out as plain text, switch from a read-level analyzer to layout or a search-oriented analyzer.
- If a field is often empty, refine its description, since the description guides the generative model.
More detail
- Extract information with multimodal pipelines that combine OCR, layout, and field extraction.
- Produce clean, grounded representations for agents and RAG with Content Understanding.
- Implement analyzers that return structured or markdown output.
Ready for the quiz?
- Which analyzer gives RAG-ready markdown with figure descriptions?
- What does enabling source and confidence estimation add to field results?
- How do you process a file that bundles several document types?
- When is Document Intelligence a better fit than Content Understanding?
Related objectives
- D5.2.S1 — Extract information by using multimodal pipelines that combine OCR, layout analysis, and field extraction
- D5.2.S2 — Produce clean, grounded representations to use with agents and RAG by using Content Understanding
- D5.2.S3 — Implement analyzers for generating structured or markdown outputs for downstream reasoning by using Content Understanding