Data Security and Governance
Audit Logging and Analysis
CoreCapture, protect, retain, centralize, and query AWS API and application evidence using CloudTrail, CloudTrail Lake, CloudWatch Logs, Athena, OpenSearch, and scalable processing.
Aligned to AWS Certified Data Engineer - Associate (DEA-C01) Version 1.1, verified August 25, 2026.
Why this matters
Audit evidence must be complete for its declared scope, protected from modification, retained appropriately, and queryable. Recent event history, durable trails, event data stores, and application logs serve different purposes.
Must Know
- Use CloudTrail to record AWS API activity with identity, time, action, source, and resource context. Select management and data events according to the audit requirement.
- Configure a trail and protected destination for long-term delivery; recent event history alone is not a complete organization-wide retention design.
- Use CloudWatch Logs for structured application and service logs with explicit retention, encryption, access, and secret filtering.
- CloudTrail Lake event data stores support governed collection, retention, and SQL querying of selected events; they are not simply another trail destination.
- Use Athena for SQL over partitioned logs in S3, Logs Insights for CloudWatch Logs, and OpenSearch where indexed ingestion and search/analytics fit.
- Large-volume EMR or distributed logs should be centralized and partitioned rather than inspected only on instance-local disks.
- Correlate application logs with CloudTrail when an audit needs both workload behavior and the actor who changed AWS configuration.
Compare and Distinguish
- CloudTrail event history versus trail versus CloudTrail Lake: recent viewing, durable delivery, and queryable event data stores differ.
- CloudTrail versus CloudWatch Logs: AWS API activity differs from application and service logs.
- Athena versus Logs Insights versus OpenSearch: query location and ingestion model determine the engine.
- Centralization versus infinite retention: durable collection still needs approved lifecycle controls.
Scenario examples
- An organization trail delivers selected API events to a protected S3 destination while CloudTrail Lake supports governed historical SQL analysis.
- Ephemeral job logs flow to CloudWatch Logs with retention and are correlated to API changes in CloudTrail.
- Archived S3 audit logs are partitioned and queried with Athena; recent application logs use Logs Insights.
Exam traps
- Relying on recent event history as complete long-term audit storage.
- Sending application debug logs and assuming they replace CloudTrail.
- Keeping credentials in logs because the destination is encrypted.
- Choosing one log query engine regardless of storage location.
Key takeaways
- Capture the right evidence at a declared scope.
- Protect and retain audit data deliberately.
- Choose query engine from source location and analysis pattern.
How it works
- Selected API and application events are delivered to protected stores with retention and access controls appropriate to their purpose.
- Correlation identifiers and aligned timestamps connect application behavior with the API actor or configuration change.
When to use it
- Use a trail for durable CloudTrail delivery, CloudTrail Lake for governed audit-event querying, and CloudWatch Logs for application logs.
- Use Athena for archived S3 logs, Logs Insights for log groups, and OpenSearch when indexed ingestion and search are required.
Security and governance implications
- Restrict who can alter, delete, or query audit stores and keep encryption and integrity controls aligned to the evidence.
- Filter secrets and unnecessary payloads at emission rather than relying on destination encryption to make them safe.
Common failure modes and diagnosis
- For an apparent audit gap, verify event type, account and Region coverage, source emission, destination permissions, and retention.
- For a failed query, confirm the engine matches the storage location and that partitions, indexes, or event data store scope include the period.
More detail
- 4.4.1: Use AWS CloudTrail to track API calls.
- 4.4.2: Use Amazon CloudWatch Logs to store application logs.
- 4.4.3: Use AWS CloudTrail Lake for centralized logging queries.
- 4.4.4: Analyze logs by using AWS services (for example, Athena, CloudWatch Logs Insights, Amazon OpenSearch Service).
- 4.4.5: Integrate various AWS services to perform logging (for example, Amazon EMR in cases of large volumes of log data).
Ready for the quiz?
- Which event source and collection scope provide the required actor, action, workload, account, Region, and time evidence?
- Should the evidence be queried in CloudWatch Logs, partitioned S3, CloudTrail Lake, or an indexed OpenSearch path?
Related objectives
- D4.4 — Task 4.4: Prepare logs for audit
- 4.4.1 — Use AWS CloudTrail to track API calls.
- 4.4.2 — Use Amazon CloudWatch Logs to store application logs.
- 4.4.3 — Use AWS CloudTrail Lake for centralized logging queries.
- 4.4.4 — Analyze logs by using AWS services (for example, Athena, CloudWatch Logs Insights, Amazon OpenSearch Service).
- 4.4.5 — Integrate various AWS services to perform logging (for example, Amazon EMR in cases of large volumes of log data).