Compute
Amazon EC2
Amazon EC2 supplies resizable virtual machines, including accelerated instances for custom model training and inference. It offers maximum operating-system and hardware control but leaves patching, scaling, and model serving to the team.
Key points
- GPU and purpose-built accelerator families address different model workloads
- Auto Scaling groups replace unhealthy instances and adjust fleet size
- Capacity Reservations or Savings Plans address availability or steady cost concerns differently
When to use it
- Run a specialized inference server that needs kernel-level tuning
- Host a licensed model appliance unavailable as a managed endpoint
Exam tips
- Choose EC2 only when low-level control justifies operational ownership; SageMaker AI manages more of the ML lifecycle
- Measure accelerator memory and token throughput, not CPU alone, when scaling LLM inference