Fundamentals of Generative AI
Generative AI Evaluation Metrics
Generative AI evaluation combines automatic metrics with human judgment. Metrics such as ROUGE, BLEU, BERTScore, and perplexity each measure different signals and do not replace safety or usefulness review.
Concepts
- ROUGE is commonly used for summarization overlap against reference summaries.
- BLEU is commonly used for machine translation precision against references.
- BERTScore compares semantic similarity using contextual embeddings.
- Perplexity measures how surprised a language model is by text; lower is generally better.
- Human evaluation is important for usefulness, safety, factuality, tone, and task fit.
Exam tips
- ROUGE is common for summarization overlap; BLEU is common for translation overlap.
- BERTScore uses contextual embeddings for semantic similarity.
- Human evaluation is important for factuality, tone, safety, and task fit.