Plan and manage an Azure AI solution
Foundry Service and Model Selection
CoreMatch each task to the right kind of model and Foundry capability: a large or small language model, a multimodal or reasoning model, a Foundry Tools service, a retrieval method, and the agent tools, memory, and knowledge sources an agent needs.
Aligned to the AI-103 study guide with skills measured as of April 16, 2026; guide and product behavior verified October 12, 2026.
Why this matters
Most AI-103 design questions start with a choice. Picking a frontier model where a small model or a Foundry Tools service would do raises cost and latency, and picking the wrong retrieval or knowledge integration leaves an agent ungrounded or exposes content that users should not see. Getting the selection right first makes every later deployment, security, and evaluation decision simpler.
Must Know
- Microsoft Foundry groups agents, models, and tools under one Azure resource provider namespace with shared role-based access control, networking, and policy. A Foundry resource holds projects, and most new development happens at project scope.
- The Foundry Models catalog includes Models sold by Azure, which Microsoft hosts and supports under Microsoft Product Terms, and models from partners and the community, which their providers support. The catalog spans large and small language models, reasoning models, multimodal models, embedding models, and domain-specific models.
- Choose a small language model for narrow, high-volume, latency-sensitive tasks; a large or reasoning model for open-ended or multistep reasoning; a multimodal model when inputs mix text with images or audio; and an embedding model to vectorize content for search.
- Foundry Tools (formerly Azure AI services) are task-specific services such as Azure Speech, Azure Language, Azure Translator, Azure Vision, Azure Content Safety, and Azure Content Understanding. Prefer them when a task has a purpose-built API with predictable, structured output.
- Model router is a single deployment that picks an underlying model for each prompt. Its routing modes are Balanced (the default), Quality, and Cost, and its rate limit and content filter apply at the router deployment.
- For retrieval, the file search tool indexes files uploaded to an agent, the Azure AI Search tool queries an existing index you manage, and a Foundry IQ knowledge base provides a shared, permission-aware knowledge layer that uses agentic retrieval over one or more knowledge sources.
- Agentic retrieval in Azure AI Search breaks a complex question into focused subqueries, runs them in parallel with semantic reranking, and returns merged grounding data with references. Classic RAG issues one hybrid query with semantic ranking, which is faster and simpler for direct questions.
- Conversation history persists in conversations managed by Foundry Agent Service, while long-term memory (preview) is configured through memory stores so an agent can recall information across sessions. Retrieval brings in enterprise knowledge, and memory carries user context forward.
- A toolbox curates tools once (for example MCP servers, Azure AI Search, file search, code interpreter, web search, and OpenAPI tools) and exposes them through one managed MCP-compatible endpoint with centralized authentication, governance, and versioning. Function calling is not hosted in a toolbox because your application runs the function.
Compare and Distinguish
- Large language model versus small language model: broad reasoning and generation versus lower cost and latency for a narrow, well-defined task.
- Multimodal model versus Foundry Tools service: flexible prompting over images, audio, and text versus a purpose-built API with a fixed output schema, such as speech to text or PII detection.
- File search versus Azure AI Search tool versus Foundry IQ: files uploaded to the agent versus an existing index you operate versus a reusable, permission-aware knowledge base shared by many agents.
- Retrieval versus memory: grounding answers in organizational content versus remembering facts about a user or task across sessions.
- Model router versus a single fixed deployment: per-prompt model selection to balance quality and cost versus one predictable model for every request.
Scenario examples
- Scenario: A help desk classifies two million short tickets a day into twelve fixed categories and needs low latency. Think: a small language model or a fine-tuned small model, not a frontier reasoning model.
- Scenario: Several agents must answer from SharePoint and Blob Storage content while respecting each user's document permissions. Think: a Foundry IQ knowledge base with permission-aware knowledge sources.
- Scenario: An agent must recall a returning customer's stated preferences in later sessions. Think: agent memory through a memory store, separate from retrieval over company documents.
Exam traps
- A bigger model is not automatically the right answer; the stem's cost, latency, and task scope usually point to a smaller model or a Foundry Tools API.
- File search is for content uploaded to the agent; it does not query an index that another team already maintains.
- Conversation history is not long-term memory; a new conversation starts without the earlier one's context unless memory or retrieval supplies it.
- Function calling cannot be shared through a toolbox, because the calling application executes the function itself.
- Model router reduces model selection work, but every routed request still counts against the router deployment's single rate limit.
Key takeaways
- Choose the smallest capability that meets the task, modality, and quality bar.
- Separate knowledge (retrieval), memory (user context), and tools (actions) when designing an agent.
- Use Foundry Tools when a purpose-built API exists, and models when the task needs open-ended reasoning or generation.
How it works
- A Foundry project endpoint gives one entry point to models, agents, and tools, so switching models rarely changes application code.
- Agentic retrieval can use a model to plan subqueries from the question and chat history before searching each knowledge source.
Objects and administrative surfaces
- Foundry portal model catalog with filters for task, deployment option, and lifecycle.
- Foundry portal Build area for agents, knowledge (Foundry IQ), and toolboxes.
- Azure AI Search for indexes, knowledge sources, and knowledge bases.
- Microsoft Foundry SDK (azure-ai-projects) and the OpenAI-compatible project endpoint.
When to use it
- Use classic single-query RAG for direct, single-topic questions where latency matters most.
- Use agentic retrieval or Foundry IQ when questions have several parts, depend on earlier turns, or span several sources.
Security and governance implications
- Models sold by Azure are hosted under Microsoft Product Terms with Microsoft support; partner and community models follow their providers' terms.
- Permission-aware knowledge sources keep agents from returning content a caller is not authorized to read.
Troubleshooting signals
- If an agent answers from general knowledge instead of company content, check that a retrieval tool or knowledge base is attached and that its instructions require citing retrieved sources.
- If costs climb on simple classification traffic, compare a smaller model or a Foundry Tools API against the current deployment.
More detail
- Select LLMs, SLMs, reasoning, multimodal, and embedding models for stated tasks.
- Choose between Foundry Tools services and prompting a general model.
- Pick classic RAG or agentic retrieval, and file search, Azure AI Search, or Foundry IQ for an agent.
- Choose memory, toolbox, MCP, OpenAPI, and function tools for agent integration.
Ready for the quiz?
- When does a small language model beat a large one for a production task?
- Which retrieval option fits an existing enterprise index, and which fits files uploaded to an agent?
- What does a Foundry IQ knowledge base add beyond a single search query?
- Why can a toolbox host MCP and OpenAPI tools but not function calling?
Related objectives
- D1.1.S1 — Choose an appropriate model for each task, including large language models (LLMs), small language models, multimodal models, and Foundry Tools
- D1.1.S2 — Choose the appropriate Foundry services for generative tasks, grounding, vector search, agent workflows, or multimodal processing
- D1.1.S3 — Choose an appropriate method for retrieval and indexing
- D1.1.S4 — Choose appropriate memory, tool, and knowledge integration services for agent solutions