beginnerintermediateadvanced

AWS · Section 20

S3 Security and Data Lifecycle

Level: intermediate
Read: 30 min
Concepts: 5

S3 access control combines identity-based IAM policies with the bucket policy itself (resource-based) — the standard way to grant another AWS account direct access. This section covers that distinction, the three encryption options (SSE-S3 now the automatic default, SSE-KMS with its separate key-level permission, and client-side), versioning and Object Lock's governance-vs-compliance retention modes, live replication's forward-only behavior versus S3 Batch Replication for existing objects, and the least-privilege discipline that keeps a public bucket a deliberate exception rather than a convenience default.

This section

Amazon S3 Data Lifecycle ManagementAWS Developers

What is true here

Identity-based policies control what an identity can attempt; bucket policies are resource-based and grant cross-account access directly.
SSE-S3 is the automatic default since Jan 2023; SSE-KMS adds auditability but requires a separate kms:Decrypt grant and, for cross-account, a customer-managed key.
Object Lock requires versioning; governance mode is bypassable with a permission, compliance mode cannot be overridden by anyone until retention expires.
Live replication (CRR/SRR) only replicates new/updated objects going forward — S3 Batch Replication handles pre-existing objects.
Default to the narrowest S3 access that satisfies the requirement — a public bucket should be a deliberate, reviewed exception.

What you will be able to do

Explain why a resource's own account must grant cross-account access via a bucket policy, not just the requester's identity policy
Debug an SSE-KMS AccessDenied by checking both the S3 policy and the KMS key policy
Choose governance vs compliance mode for Object Lock based on whether the retention policy needs to remain adjustable
Recognize when a replication gap is caused by pre-existing objects and needs S3 Batch Replication
Apply least privilege by default and treat a public bucket as a reviewed exception

From S3 access control to a least-privilege default

Identity vs bucket policy

SSE-S3, SSE-KMS, client-side

Versioning, Object Lock, lifecycle

Replication, cross-account

Least privilege default

S3 Security and Data Lifecycle

Identity vs resource-based access, encryption options, versioning/Object Lock/lifecycle, replication, and the least-privilege discipline.

Identity-Based vs Resource-Based S3 Access

coreintermediate

An identity-based policy is attached to an IAM user/role and says what that identity can do. A bucket policy is attached to the bucket itself (resource-based) and says who can do what to it — including granting access to a completely different AWS account. Both are evaluated together; access requires no explicit deny and at least one explicit allow.

Think of it as

An identity-based policy is a badge you personally carry ("I can open any door labeled finance"). A bucket policy is a sign on the door itself ("only badges from accounts A and B may enter"). Getting through requires satisfying both — your badge and the door's own rule.

json

{ "Effect": "Allow", "Principal": { "AWS": "arn:aws:iam::222233334444:root" }, "Action": "s3:GetObject", "Resource": "arn:aws:s3:::my-bucket/*" }

What we're doing: See a bucket policy grant cross-account read access without any role assumption.

bucket-policy.jsonjson

{
  "Statement": [{ "Effect": "Allow", "Principal": { "AWS": "arn:aws:iam::222233334444:root" },
    "Action": "s3:GetObject", "Resource": "arn:aws:s3:::shared-bucket/*" }]
}

2: The Principal names a different AWS account (222233334444) directly — that account's users/roles with s3:GetObject permission can now read this bucket, with no cross-account role assumption needed on either side.

Why this works: This is the specific capability identity-based policies alone cannot provide — an identity-based policy can only grant permissions to act on resources, it cannot itself make a resource accessible to a principal in another account the way a resource-based policy can.

Trying to grant cross-account S3 access with only an identity-based policy

Wrong

text

# Attach an identity-based policy in Account A granting s3:GetObject on
# Account B's bucket, and expect Account B's users to now have access

Better

text

# Account B must ALSO add a bucket policy granting Account A's principal
# access — Account A's own policy alone cannot open Account B's bucket

What you see: A user in Account A still gets AccessDenied when reading an object in Account B's bucket, despite Account A's IAM policy explicitly granting s3:GetObject on that bucket's ARN.

Why: An identity-based policy only controls what the identity is permitted to attempt — it cannot grant access on a resource it does not own; the resource's own account must independently grant that access via a bucket policy (or ACL), since both sides of a cross-account request must agree.

The badge vs the sign on the door

Identity-based policy

+Attached to an IAM user/role
+Says what that identity can attempt
+Cannot alone grant cross-account access

Bucket policy

−Attached to the bucket itself
−Can name a principal in another account directly
−The standard way to grant cross-account access

Identity-based vs bucket (resource-based) policy

Identity-based vs bucket (resource-based) policy
Property	Identity-based	Bucket policy
Attached to	IAM user/role	The bucket itself
Grants cross-account access directly?	No — needs a role to assume	Yes — can name another account's principal
Scope	Whatever resources it references	This bucket and its objects
Size limit	Standard IAM policy limits	20 KB

Remember: Identity-based policies are attached to IAM users/roles; bucket policies are attached to the bucket and are the standard way to grant cross-account access directly. Both are evaluated together — access needs no explicit deny and at least one explicit allow.

S3 Encryption Options

coreintermediate

Since January 2023, S3 automatically encrypts every new object with SSE-S3 by default, at no extra cost. SSE-KMS is an opt-in alternative using a KMS key you can rotate, audit, and control cross-account sharing for — at extra KMS request cost. Client-side encryption encrypts data before it ever reaches S3, so AWS never sees the plaintext at all.

Think of it as

SSE-S3 is the building's own lock, already installed and free. SSE-KMS is bringing your own lock whose key AWS still holds and applies for you, but which you control access to and can audit. Client-side encryption is locking the box yourself before handing it to the building at all — the building never even has a key that opens it.

bash

--server-side-encryption AES256          # SSE-S3
--server-side-encryption aws:kms --ssekms-key-id <key>   # SSE-KMS

What we're doing: See why an SSE-KMS-encrypted object needs a second, separate permission from the ordinary S3 policy.

sse-kms-access.jsonjson

S3 bucket policy: grants s3:GetObject on the object  ✓
KMS key policy: does NOT grant kms:Decrypt to this principal  ✗
→ GetObject still fails — both permissions are required together

2: The S3-level permission alone is not sufficient — decrypting an SSE-KMS object requires kms:Decrypt on the specific KMS key that encrypted it.
3: Without that second grant, the request is denied even though the S3 bucket policy looks completely correct on its own.

Why this works: SSE-KMS deliberately layers two independent authorization checks — S3 access and KMS key access — so that having one without the other is not enough, which is exactly the point when a key needs tighter control than the bucket policy alone provides.

Debugging an SSE-KMS AccessDenied by only checking the S3 bucket policy

Wrong

text

# "The bucket policy clearly grants GetObject — this AccessDenied makes no sense."

Better

text

# Check the KMS key policy too — SSE-KMS objects require BOTH an S3-level
# grant AND a KMS-level grant (kms:Decrypt) on the specific key

What you see: Time is spent re-reviewing an S3 bucket policy that is already correct, while the actual missing grant is on the KMS key policy for the key that encrypted the object.

Why: SSE-KMS access failures often have nothing wrong in the S3 policy at all — the KMS key policy is a completely separate, independently-enforced permission boundary that many engineers forget to check first.

Who holds the key, and how much control you get

auditable / shareableno setup

AWS manages ityou control it

SSE-S3: automatic default, free
SSE-KMS: auditable, rotatable, cross-account (customer-managed key)
Client-side: AWS never sees plaintext

Choosing an S3 encryption option

Choosing an S3 encryption option
Need	Choice
Baseline encryption at rest, no setup	SSE-S3 (automatic default)
Auditable key usage, rotation control	SSE-KMS
Cross-account object sharing with a scoped key	SSE-KMS with a customer-managed key
AWS must never see plaintext at all	Client-side encryption

Together

bash

aws s3api put-object --bucket my-bucket --key file.txt --body file.txt --server-side-encryption aws:kms --ssekms-key-id alias/my-key

Remember: SSE-S3 is the automatic default (free, no setup) since Jan 2023. SSE-KMS adds auditability/rotation and requires a separate kms:Decrypt grant beyond the S3 policy — and only a customer-managed key supports cross-account sharing. Client-side encryption keeps AWS from ever seeing plaintext.

Versioning, Object Lock, and Lifecycle Transitions

coreintermediate

Versioning keeps every past copy of an object under distinct version IDs. Object Lock (which requires versioning) adds WORM protection on top — governance mode can be bypassed with a special permission, compliance mode cannot be overridden by anyone, including the root user, until the retention period expires. Lifecycle rules automate storage-class transitions and expiration over time.

Think of it as

Versioning is keeping every draft of a document instead of overwriting it. Object Lock is putting a specific draft in a safe with a timer — governance mode is a safe the building manager can still open in an emergency; compliance mode is a safe that genuinely cannot be opened by anyone before the timer runs out.

What we're doing: See why a compliance-mode retention period cannot be shortened, even by an account administrator.

compliance-mode.txttext

Object locked: compliance mode, retain until 2027-01-01
Admin (root account) attempts to delete the object early
→ Access Denied — compliance mode has no override, by design

1: The retention date is stored in the object version's own metadata at lock time.
3: Compliance mode is specifically designed with no bypass mechanism at all — not even the root account can shorten or remove it, which is the entire point for regulatory WORM requirements.

Why this works: This "no override, not even for root" property is precisely what makes compliance mode usable for regulatory requirements (SEC 17a-4, FINRA, CFTC) that specifically require data to be provably un-deletable by anyone, including insiders.

Choosing compliance mode for a policy the team might need to test or adjust

Wrong

text

# Lock objects in compliance mode immediately, planning to "adjust the
# retention period later if it turns out to be wrong"

Better

text

# Use governance mode first to validate the retention policy in practice —
# only switch to compliance mode once the policy is confirmed correct

What you see: A retention period turns out to be set incorrectly (too long, wrong objects), and there is no way to fix it — the objects are locked exactly as configured until the retention date, full stop.

Why: Compliance mode's defining property — no override by anyone — applies just as strictly to an honest configuration mistake as it does to a malicious deletion attempt, which is why AWS's own guidance recommends validating retention behavior in governance mode first.

Governance mode vs compliance mode

Governance mode

+Protects against deletion/overwrite by most users
+Bypassable with s3:BypassGovernanceRetention
+Good for testing a retention policy before compliance mode

Compliance mode

−Cannot be overridden by anyone, including root
−Retention period cannot be shortened once set
−Only way to delete early: delete the entire AWS account

Remember: Object Lock requires versioning. Governance mode is bypassable with a special permission (good for testing); compliance mode has no override for anyone, including root, until the retention period expires — choose deliberately, since compliance mode cannot be walked back.

standardintermediate

Live replication (Cross-Region or Same-Region) automatically copies new/updated objects going forward — it does not retroactively copy objects that existed before replication was configured. S3 Batch Replication handles that on-demand for existing objects. Replication can also change object ownership on copy, which is how cross-account sharing stays clean.

Think of it as

Live replication is a photocopier that only starts running the moment you turn it on — everything already in the filing cabinet before that stays uncopied unless you separately run a batch job over the old files too.

text

Live replication (CRR/SRR): new + updated objects only, going forward
S3 Batch Replication: on-demand, for existing/failed/already-replicated objects

What we're doing: See why enabling replication on a bucket with years of existing data does not back-copy anything by itself.

replication-gap.txttext

Bucket has 3 years of existing objects
Live replication (CRR) enabled today
→ Only objects uploaded/modified AFTER today start replicating —
  the 3 years of pre-existing objects need a separate Batch Replication job

1: Every object already in the bucket predates the replication rule's activation.
3: Live replication only reacts to new writes going forward — closing the gap for existing objects requires explicitly running S3 Batch Replication as a separate, on-demand job.

Why this works: This is a common source of "why isn't my replica complete" confusion — live replication was designed as a forward-looking, event-driven mechanism, not a one-time full sync, which is precisely the gap Batch Replication exists to fill.

Assuming enabling replication retroactively copies all existing objects

Wrong

text

# "We turned on CRR, so the destination bucket should now have everything
# from the source bucket."

Better

text

# Run an S3 Batch Replication job specifically to replicate the
# pre-existing objects — live replication alone will not touch them

What you see: A disaster-recovery destination bucket is missing years of historical data, discovered only during an actual recovery attempt, because replication was assumed to have been a full sync from day one.

Why: Live replication is explicitly and only a forward-looking mechanism triggered by new writes — verifying replica completeness requires knowing this distinction and running Batch Replication for anything written before the rule existed.

Remember: Live replication (CRR/SRR) only copies new/updated objects going forward — use S3 Batch Replication for pre-existing objects. CRR crosses Regions (compliance, latency); SRR stays in-Region (log aggregation, data sovereignty). Owner override cleanly separates replica ownership for cross-account sharing.

Least Privilege and Avoiding Public Buckets

standardintermediate

Default to the narrowest S3 access that satisfies the requirement — scope bucket policies to specific principals and actions, keep Block Public Access enabled, and treat a public bucket as a deliberate, reviewed exception rather than a convenience default.

Think of it as

Every grant of access, once made, is a standing liability that has to be remembered and eventually revisited — starting narrow and widening only when a specific, understood need appears is cheaper than starting wide and trying to claw permissions back later.

text

Default: private, narrowest policy that satisfies the requirement.
Public: only as a deliberate, reviewed exception — e.g. static asset bucket behind CloudFront.

What we're doing: Choose the narrowest mechanism for a case that looks like it needs a public bucket but does not.

decision.txttext

Requirement: let a mobile app download user-specific report PDFs
→ Not a public bucket — presigned URLs scoped to one object, short-lived

Requirement: serve a public marketing site's static assets
→ CloudFront + S3, with Origin Access Control — not a directly public bucket

1: User-specific files are exactly the case that looks like it needs public access but actually needs per-request, scoped, short-lived access instead.
3: Even genuinely public content is usually served through CloudFront with Origin Access Control, keeping the bucket itself private and only CloudFront able to read from it directly.

Why this works: Both cases initially look like "we need public access," but in each one a narrower mechanism (presigned URLs, or CloudFront OAC) satisfies the real requirement without leaving the bucket itself broadly or permanently exposed.

Remember: Default to the narrowest S3 access that satisfies the requirement — scoped bucket policies, presigned URLs, or access points over wildcard Principals. A public bucket should be a deliberate, reviewed exception, and even then usually sits behind CloudFront rather than being directly public.