Implementation and Integration
Model Deployment, Capacity, and Resilience
Choose on-demand, provisioned, batch, serverless, container, or SageMaker endpoint patterns based on traffic shape, latency, model ownership, GPU needs, availability, and cost.
Concepts
- On-demand APIs suit variable traffic; provisioned capacity suits predictable sustained throughput and latency.
- Self-hosted models require capacity planning for model loading, GPU memory, token throughput, scaling, and patching.
- Canary releases and rollback criteria reduce risk when models, prompts, adapters, or inference containers change.
- Cross-Region inference and fallback models can improve resilience when supported by data residency requirements.
- AgentCore Runtime provides isolated serverless execution for custom-framework agents, including extended asynchronous workloads; durable business job state still belongs in application storage.
Exam tips
- Bedrock Provisioned Throughput reserves model capacity; it is not a SageMaker autoscaling feature.
- Use SageMaker AI endpoints for hosted custom models that need endpoint-level deployment control.
- Model Registry tracks approved model versions and deployment status.