Database
Amazon ElastiCache
Amazon ElastiCache provides managed Valkey, Redis OSS, or Memcached-compatible in-memory caching. It reduces repeated model and retrieval work through low-latency response, semantic, session, and rate-limit caches.
Key points
- Valkey and Redis-compatible engines offer rich structures and replication
- Memcached emphasizes simple distributed caching
- Eviction, TTL, encryption, and failover choices affect correctness as well as speed
When to use it
- Cache authorized repeated LLM responses
- Maintain a low-latency semantic cache or distributed token bucket
Exam tips
- Choose ElastiCache for hot ephemeral data and DynamoDB for durable key-value state
- A semantic cache key must isolate tenants, prompt and model versions, permissions, and freshness