beginnerintermediateadvanced

AWS · S3 — Object Storage

S3 — Object Storage

The object model, consistency and durability, large-file patterns, public access risk, and real workload shapes

Bucket → object → keyno real folders, only shared prefixes
Strongly consistent PUT/DELETEdurability ≠ availability — separate guarantees
Multipart: 5 MiB–5 GiB partsup to 10,000 parts, 48.8 TiB max object
Block Public Access overrideson by default, keep it that way

1.S3 Core Vocabulary

A bucket is a container; an object (data + metadata) lives inside it, addressed by a unique key. There is no real folder structure — a "prefix" like photos/ is just a shared substring in keys, S3 fakes a folder view over a flat namespace. A bucket policy or IAM policy controls access; versioning keeps every past copy of an object; lifecycle rules move or expire objects automatically over time.

Key features

Object = data + metadata; key = the unique identifier for that object within a bucket
A "folder" is not real — a prefix is a shared substring convention over a flat key namespace
Storage classes trade cost against retrieval latency/frequency: Standard, Standard-IA, One Zone-IA, Glacier tiers
Versioning keeps every PUT/DELETE as a distinct version, keyed by a generated version ID
Lifecycle rules automate transitions between storage classes or expiration, based on object age/prefix
A bucket policy is a resource-based IAM policy; by default all buckets and objects are private

S3 vocabulary, container to controls

Bucket

the container, globally unique name

Object

data + metadata, addressed by a key

Key / prefix

the unique identifier — prefixes fake folders

Bucket policy / IAM

private by default, access explicitly granted

Signature

bucket → key (object) — no real folders, only shared prefixes

Bucket = container, object = data + metadata, key = unique identifier. "Folders" are a prefix illusion over a flat namespace — a folder rename is really a copy-then-delete of every object under that prefix. Private by default; verify, don't assume.

2.Consistency, Durability, and Availability

S3 provides strong read-after-write consistency for PUT and DELETE in every Region — a read immediately following a successful write returns the new data, not a stale one. Durability (objects surviving over time) and availability (being reachable right now) are different guarantees. S3 is not a POSIX filesystem: no real directories, no atomic rename, no partial-file appends.

Key features

Strong read-after-write consistency for PUT/DELETE, in all Regions — no "eventual consistency" window for object data
Updates to a single key are atomic — concurrent reads never see partial/corrupt data, only old or new
Bucket-level configuration changes (e.g. enabling versioning) have an eventual consistency model, unlike object data
Durability ≠ availability: durability is about not losing data over time; availability is about being reachable right now
S3 is not POSIX: no real directories, no atomic rename, no partial in-place file edits — only whole-object PUT/GET/DELETE

Three separate, independent guarantees

is it correctcan I reach it

about the pastabout right now

Durability: will the data still exist a year from now
Availability: can I reach it right this second
Consistency: does a read see the latest write, right now

Command

text

PUT/DELETE on an object key: strongly consistent, all Regions
Bucket-level config changes: eventually consistent — allow time to propagate

S3 gives strong read-after-write consistency for object PUT/DELETE in all Regions — no eventual-consistency window for data. Durability (data survives) and availability (reachable now) are separate guarantees. S3 is not POSIX: no real directories, no atomic rename, no partial writes.

3.Multipart Upload and Large-File Patterns

Multipart upload splits one object into independently-uploaded parts (5 MiB–5 GiB each, up to 10,000 parts, max object size 48.8 TiB) that S3 assembles afterward — recommended once an object approaches 100 MB. Range requests (GET with a Range header) fetch part of an object without downloading the whole thing.

Key features

Multipart upload: parts are 5 MiB–5 GiB each (no minimum on the last part), up to 10,000 parts, max object size 48.8 TiB
AWS recommends multipart upload once an object approaches ~100 MB
A failed part can be retried individually — the whole object does not need to restart
Range GET requests fetch a byte range of an object without downloading the whole object
Object metadata (system + custom key-value pairs) and tags travel with the object; tags can also drive lifecycle rules and access policies

Multipart upload core specifications

Multipart upload core specifications
Item	Value
Maximum object size	48.8 TiB
Maximum parts per upload	10,000
Part size	5 MiB – 5 GiB (no minimum on the last part)

Command

bash

aws s3api create-multipart-upload --bucket my-bucket --key large-file.zip
aws s3api upload-part --bucket my-bucket --key large-file.zip --part-number 1 --upload-id <id> --body part1

Multipart upload: 5 MiB–5 GiB parts (no minimum on the last), up to 10,000 parts, max 48.8 TiB object — recommended once an object nears 100 MB. Each part retries independently. Range GETs fetch part of an object without a full download.

4.Public Access Risks and Block Public Access

S3 Block Public Access is a bucket-level (or account-level) setting, on by default, that overrides any bucket policy or ACL trying to grant public access. AWS recommends keeping all four Block Public Access settings enabled unless a specific, deliberate use case requires public objects.

Key features

Block Public Access has four independent settings, controllable at the account or bucket level
On by default for new buckets — AWS recommends leaving all four enabled unless a use case explicitly requires public objects
Block Public Access overrides bucket policies and ACLs that would otherwise grant public access
A public object exposure is usually the combination of Block Public Access being disabled AND a policy/ACL granting public read — either alone is often not enough

Command

bash

aws s3api get-public-access-block --bucket my-bucket
aws s3api put-public-access-block --bucket my-bucket --public-access-block-configuration BlockPublicAcls=true,IgnorePublicAcls=true,BlockPublicPolicy=true,RestrictPublicBuckets=true

Block Public Access overrides bucket policies/ACLs that would grant public access — on by default, keep it enabled unless a use case genuinely requires public objects. Real public exposure needs BOTH BPA disabled AND a policy/ACL actually granting access.

5.Common S3 Workload Patterns

The same S3 primitives (buckets, storage classes, lifecycle rules, event notifications, presigned URLs) combine into different shapes depending on the workload: static assets fronted by CloudFront, direct client uploads via presigned URLs, scheduled backups with lifecycle transitions to cold storage, and a data lake where S3 is the durable, queryable storage layer under services like Athena or EMR.

Key features

Static assets: S3 + CloudFront for cached, edge-delivered content, often with an Origin Access Control restricting direct bucket access
Uploads: presigned URLs let a client (browser, mobile app) upload directly to S3 without routing the bytes through your backend
Backup/archival: lifecycle rules transition objects Standard → Standard-IA → Glacier tiers as they age, cutting storage cost
Data lake: S3 as the durable storage layer, queried in place by Athena/EMR/Redshift Spectrum rather than being loaded into a database first

Command

bash

aws s3 presign s3://my-bucket/uploads/file.jpg --expires-in 300

The same S3 primitives combine differently per workload: static assets → S3 + CloudFront, uploads → presigned URLs (bypass the backend), backup/archival → lifecycle rules across storage classes, data lake → S3 queried in place by Athena/EMR.

Spotted a mistake on this page? Tell us.

Read Sheet