Machine Learning
Amazon Bedrock Model Evaluation
Amazon Bedrock evaluations support automated, LLM-judge, and human assessment of models plus retrieve-only and retrieve-and-generate evaluation of knowledge bases and other RAG sources. They help compare versions, but production decisions still need application-specific safety, latency, cost, and business evidence.
Key points
- Evaluation jobs reference explicit models, prompts, datasets, and output locations
- Human workflows capture rubric-based judgments
- LLM-as-a-judge results should be calibrated against qualified human labels
- RAG evaluation separates context relevance and coverage from answer correctness and faithfulness
When to use it
- Compare candidate models on representative contract summaries
- Gate a prompt release on quality and safety regression thresholds
Exam tips
- Choose Model Evaluation for FM output assessment and SageMaker Clarify for supported bias or explainability analysis
- Use AgentCore Evaluations for trace-based agent task and tool behavior; use Bedrock RAG evaluations for retrieval and grounded-generation evidence
- Keep a sealed holdout set because repeated tuning against one benchmark overfits the release process