Machine Learning
Amazon SageMaker Neo
Amazon SageMaker Neo compiles supported trained models for optimized inference on selected cloud and edge hardware. It reduces runtime footprint or latency when a model-framework-target combination is supported.
Key points
- Compilation targets a specific framework and hardware environment
- Generated artifacts run through the Neo runtime
- Unsupported model operators or dynamic behavior may prevent compilation
When to use it
- Optimize a compact vision model for an edge device
- Reduce latency for a supported inference workload on designated instances
Exam tips
- Choose Neo only after profiling shows compilation can address the bottleneck
- Validate accuracy after compilation and keep the uncompiled artifact available for rollback