Fundamentals of Generative AI
AWS Generative AI Infrastructure and Cost Tradeoffs
AWS infrastructure choices for generative AI trade off responsiveness, availability, regional coverage, data residency, and cost. For AIF-C01, know how token-based pricing, streaming, cross-Region inference, batch inference, and Provisioned Throughput change cost and performance.
Concepts
- On-demand Amazon Bedrock inference is billed per input and output token, so prompt and response length drive cost and latency.
- Streaming operations such as ConverseStream return output as it is generated, which improves perceived responsiveness without changing total tokens.
- Cross-Region inference routes requests across AWS Regions; geographic profiles keep processing within a geography such as the EU, while global profiles route worldwide at lower cost.
- Provisioned Throughput reserves model capacity for an hourly commitment, and batch inference lowers cost for workloads that do not need immediate responses.
Exam tips
- Choose a geographic cross-Region inference profile when data residency rules apply; choose global for cost savings without geographic limits.
- Streaming improves time to first output, not total cost.
- Match the pricing option to the traffic pattern: on-demand for variable, Provisioned Throughput for steady, batch for offline.