Data Security and Governance
Authorization and Least Privilege
CoreBuild custom least-privilege policies, select credential stores, grant database and Lake Formation permissions, and apply role- or attribute-based authorization at the correct layer.
Aligned to AWS Certified Data Engineer - Associate (DEA-C01) Version 1.1, verified August 25, 2026.
Why this matters
Authentication identifies a caller; authorization decides what that caller can do. Data workloads often cross IAM, resource, catalog, storage, and database boundaries, and success requires only the necessary grants at each.
Must Know
- Create a custom IAM policy when a managed policy is broader or otherwise unsuitable. Scope actions, resources, and conditions to the workload and test expected denies.
- Least privilege is not merely removing AdministratorAccess; it requires the exact read, write, list, decrypt, pass-role, or service actions at correct resource scopes.
- Use Secrets Manager for rotating secrets and Parameter Store for configuration or secure parameters when its lifecycle and integrations fit. Avoid source files and shared plaintext variables.
- Database authentication does not automatically grant SQL object privileges. Use database users, groups, or roles to grant the required schemas, tables, and operations.
- Lake Formation permissions govern cataloged data for integrated engines. Metadata visibility and S3 access alone might still be insufficient or incorrectly scoped.
- Use role-based access for stable job duties and tag- or attribute-based conditions when scalable contextual policy depends on principal, resource, request, or classification attributes.
- Network allowlists and encryption keys do not substitute for authorization.
Compare and Distinguish
- Managed versus custom policy: convenience and reuse differ from workload-specific least privilege.
- Secrets Manager versus Parameter Store: rotation-oriented secret lifecycle differs from parameter and configuration storage.
- IAM permission versus database grant versus Lake Formation permission: AWS API, SQL object, and governed lake access are distinct.
- Role-based versus attribute-based authorization: stable duty membership differs from policy evaluation on context.
Scenario examples
- A Glue role reads one input prefix and writes one output prefix through a custom policy without wildcard bucket access.
- Analysts authenticate successfully to Redshift, then receive database-role grants only on approved schemas.
- A Lake Formation principal receives table permissions plus the required underlying data-location access.
Exam traps
- Copying a broad managed policy and calling it custom least privilege.
- Using IAM AdministratorAccess to solve a missing SQL GRANT.
- Assuming catalog visibility proves governed table access.
- Treating tags as an authentication mechanism.
Key takeaways
- Authorize at the layer where the protected action occurs.
- Combine only the necessary identity, resource, catalog, storage, and database permissions.
- Use custom conditions to narrow access, not to hide broad wildcards.
How it works
- A request is evaluated against identity and resource policies, explicit denies, conditions, and any service-specific data permissions.
- Custom policies narrow read, write, list, decrypt, or pass-role actions to the durable resource boundary needed by the task.
When to use it
- Use a custom policy when a managed policy is broader than the workload, and use role-based access for stable job duties.
- Use attribute or tag conditions when access legitimately depends on principal, resource, request, or classification context.
Security and governance implications
- Keep secrets out of source and shared variables, and select Secrets Manager or Parameter Store from the required lifecycle.
- Test allowed and denied paths after every policy change, including cross-account and KMS-dependent operations.
Common failure modes and diagnosis
- For AccessDenied, inspect the exact action, resource ARN, condition context, explicit denies, and resource-side policy.
- If authentication succeeds but data access fails, trace database, Lake Formation, S3, and KMS permissions independently.
More detail
- 4.2.1: Create custom IAM policies when a managed policy does not meet the needs.
- 4.2.2: Store application and database credentials (for example, Secrets Manager, AWS Systems Manager Parameter Store).
- 4.2.3: Provide database users, groups, and roles access and authority in a database (for example, for Amazon Redshift).
- 4.2.4: Manage permissions through AWS Lake Formation (for Amazon Redshift, Amazon EMR, Amazon Athena, and Amazon S3).
- 4.2.5: Apply authorization methods that address business needs (role-based, tag-based, and attribute-based).
- 4.2.6: Construct custom policies that meet the principle of least privilege.
Ready for the quiz?
- What exact actions and resources does the workload need, and which explicit deny or resource policy could still block them?
- Is the missing permission in IAM, a database grant, Lake Formation, KMS, or another independent authorization layer?
Related objectives
- D4.2 — Task 4.2: Apply authorization mechanisms
- 4.2.1 — Create custom IAM policies when a managed policy does not meet the needs.
- 4.2.2 — Store application and database credentials (for example, Secrets Manager, AWS Systems Manager Parameter Store).
- 4.2.3 — Provide database users, groups, and roles access and authority in a database (for example, for Amazon Redshift).
- 4.2.4 — Manage permissions through AWS Lake Formation (for Amazon Redshift, Amazon EMR, Amazon Athena, and Amazon S3).
- 4.2.5 — Apply authorization methods that address business needs (role-based, tag-based, and attribute-based).
- 4.2.6 — Construct custom policies that meet the principle of least privilege.