Data Store Management
Data Lifecycle, Retention, and Resilience
CoreMove data between S3 and Redshift, transition or expire S3 objects, manage versions and DynamoDB TTL, satisfy deletion duties, and preserve availability within sovereignty constraints.
Aligned to AWS Certified Data Engineer - Associate (DEA-C01) Version 1.1, verified August 25, 2026.
Why this matters
Lifecycle settings encode both cost and policy. A transition is not deletion, asynchronous expiry is not a deadline, and resilience copies can conflict with legal deletion or Region restrictions.
Must Know
- Use Redshift COPY for bulk S3-to-Redshift loading and UNLOAD for Redshift-to-S3 export, with an authorized role, format, and destination.
- An S3 Lifecycle transition changes storage class after an age or condition; expiration removes current or noncurrent versions according to the rule.
- S3 Versioning preserves object versions for recovery. Deleting a current object can create a delete marker rather than remove every version.
- DynamoDB TTL uses an item attribute to mark expiration and removes expired items asynchronously; it is not a synchronous deletion guarantee.
- A legal deletion workflow must identify live objects or rows, noncurrent versions, replicas, backups, derived data, and catalog references within the approved scope.
- Select versioning, backup, replication, multi-AZ behavior, and recovery testing from RPO/RTO while respecting allowed accounts and Regions.
- Order transition and expiration actions so archival savings do not cause premature deletion or indefinite retention.
Compare and Distinguish
- S3 transition versus expiration versus versioning: storage-class movement, age-based deletion, and overwrite recovery are independent controls.
- DynamoDB TTL versus explicit delete: asynchronous aging differs from confirmed removal by a deadline.
- Resilience versus deletion: extra copies aid recovery but expand the surfaces that policy must govern.
- COPY versus UNLOAD: warehouse load direction differs from export direction.
Scenario examples
- Raw S3 data transitions after 30 days and expires after 365 days under separate lifecycle actions.
- A deletion request removes approved live and historical copies and produces evidence, rather than depending on DynamoDB TTL alone.
- Redshift loads curated Parquet through COPY and exports governed results through UNLOAD.
Exam traps
- Using an archival transition to satisfy deletion.
- Treating a delete marker as removal of every object version.
- Depending on TTL for a fixed legal deadline.
- Adding cross-Region replication without checking location restrictions.
Key takeaways
- Model cost, retention, deletion, and resilience together.
- Distinguish asynchronous aging from verified deletion.
- Apply lifecycle rules to the intended version and replication scope.
How it works
- Lifecycle rules evaluate object state and age for transition or expiration, while versioning preserves historical object versions.
- Recovery design creates and tests approved copies; deletion workflows enumerate those copies and record their final state.
When to use it
- Use versioning for overwrite recovery, backup for retained recovery points, and replication for an explicitly allowed continuity design.
- Use DynamoDB TTL for asynchronous aging of temporary items, but use explicit deletion and verification for a fixed deadline.
Security and governance implications
- Keep lifecycle roles, backup vaults, replicas, KMS keys, and restore permissions within approved accounts and Regions.
- Apply legal holds and retention controls before changing expiration behavior, and restrict who can weaken those policies.
Common failure modes and diagnosis
- For objects that remain, inspect current and noncurrent version rules, delete markers, replication, retention, and legal holds.
- For failed Redshift movement, verify COPY or UNLOAD direction, role permissions, source or destination path, and data format.
More detail
- 2.3.1: Perform load and unload operations to move data between Amazon S3 and Amazon Redshift.
- 2.3.2: Manage S3 Lifecycle policies to change the storage tier of S3 data.
- 2.3.3: Expire data when it reaches a specific age by using S3 Lifecycle policies.
- 2.3.4: Manage S3 versioning and DynamoDB TTL.
- 2.3.5: Delete data to meet business and legal requirements.
- 2.3.6: Protect data with appropriate resiliency and availability.
Ready for the quiz?
- Does the requirement call for a storage-class transition, age-based expiration, overwrite recovery, or verified deletion?
- Which replicas, versions, backups, and derived copies must be included without crossing an allowed account or Region boundary?
Related objectives
- D2.3 — Task 2.3: Manage the lifecycle of data
- 2.3.1 — Perform load and unload operations to move data between Amazon S3 and Amazon Redshift.
- 2.3.2 — Manage S3 Lifecycle policies to change the storage tier of S3 data.
- 2.3.3 — Expire data when it reaches a specific age by using S3 Lifecycle policies.
- 2.3.4 — Manage S3 versioning and DynamoDB TTL.
- 2.3.5 — Delete data to meet business and legal requirements.
- 2.3.6 — Protect data with appropriate resiliency and availability.