API Reliability and Idempotency on AWS cheat sheet · AWS

1.What Idempotency Actually Means

An idempotent operation can be retransmitted or retried with no additional side effects. Running it twice leaves the system in the same state as running it once, and the caller gets the same answer either way. That property is what makes retries safe — and without it, every timeout becomes a choice between losing work and duplicating it.

Key features

AWS defines an idempotent operation as one where a request can be retransmitted or retried with no additional side effects
AWS's preferred approach is to incorporate a unique caller-provided client request identifier into the API contract — EC2 exposes this as `ClientToken`
A retry must return a semantically equivalent response, so the caller cannot tell whether it was the first attempt or the fifth
Recording the idempotency token and performing the mutation must be one atomic, consistent, isolated and durable operation, or partial state results
The service has to keep request history beyond the lifetime of the resource, because a late retry can arrive after the resource is gone
Idempotency is a property of the operation as a whole — a handler that writes two rows in two transactions is not idempotent even if each write is

Where the ambiguous timeout shows up

Where the ambiguous timeout shows up
Place	Why a retry happens	What a duplicate costs
A public API	The client, a proxy, or a mobile network retries	Two orders, two accounts, two of whatever was created
A payment operation	The gateway times out; the caller retries	A customer charged twice — the most expensive duplicate there is
An SQS consumer	The visibility timeout expires before the handler finishes	The message is processed again by another worker
An asynchronous Lambda	Lambda retries the invocation on error	The handler runs twice for one event
An event source mapping	A batch is redelivered after a partial failure	Every record in the batch is reprocessed, not just the failed one
A distributed workflow	A step is retried after a transient error	That step's effect is applied twice mid-workflow

The ambiguous timeout, and what it forces

The client sees one thing — no response — in three completely different situations. Idempotency is what makes the same next action correct in all three.

Signature

client-supplied key + atomic record-and-mutate + stored response on retry + retention beyond resource lifetime

An idempotent operation can be retried with no additional side effects, because the client cannot tell a lost response from a lost request. Take a client-supplied key, record it and perform the mutation in one atomic transaction, return the same stored response to every retry, and keep the record longer than the resource itself lives.

2.The Mechanisms, and Which One to Reach For

There are five ways to make an operation safe to repeat, and they are not interchangeable. An idempotency key identifies the request; a conditional write makes the first attempt win atomically; deduplication filters repeats within a window; a unique constraint lets the database enforce it; durable state remembers what was already done. Most real designs use two of them together.

Key features

A DynamoDB conditional write with `attribute_not_exists(pk)` succeeds only when no item with that key exists; when it fails the response is "The conditional request failed"
SQS FIFO queues do not introduce duplicate messages: retrying `SendMessage` within the 5-minute deduplication interval adds no duplicate
FIFO deduplication is either content-based — a SHA-256 hash of the message body, excluding attributes — or an explicit message deduplication ID
S3 replaces an existing object when the key already exists, so a deterministic object key is itself an idempotency mechanism
A unique constraint or a conditional write moves the race into the storage engine, where it is actually atomic
Deduplication windows are bounded — SQS FIFO's is five minutes — so anything that can be retried later needs durable state instead

The five mechanisms, side by side

The five mechanisms, side by side
Mechanism	Enforced by	Window	Best for
Idempotency key	Your code plus a unique index	As long as you retain it	Creates over a public API, where no natural key exists
Conditional write	The database, atomically	As long as the item exists	DynamoDB writes, claims, and optimistic concurrency
Unique constraint	The database, atomically	As long as the row exists	Relational creates with a natural key
SQS FIFO deduplication	SQS	5-minute interval	Producer-side retries of the same send
Deterministic object key	S3 replace-on-write	Unbounded	File and artifact processing pipelines
Durable state	Your storage	Whatever you choose	Anything retried after a bounded window closes

Producer-side and consumer-side are different problems

Producer-side and consumer-side are different problems
Aspect	Producer-side duplicate	Consumer-side duplicate
Cause	The same message is sent twice after a timeout	One message is delivered or processed more than once
Fixed by	SQS FIFO deduplication, within 5 minutes	An idempotent handler — nothing else
Bounded?	Yes, by the deduplication interval	No — a redelivery can happen at any time
Standard queues	No deduplication at all	Same requirement: the handler must be idempotent
Common error	Assuming FIFO makes the consumer safe too	Assuming a visibility timeout prevents reprocessing

Five mechanisms, and where each one belongs

The choice is usually made for you by what identifies the work: a natural key means the storage layer can enforce it, and only the absence of one requires a caller-supplied token.

Signature

idempotency key · conditional write · unique constraint · FIFO dedup (5 min) · deterministic key · durable state

Let the storage engine enforce it whenever a natural key exists — a conditional write or unique constraint is atomic and free. Take a client-supplied idempotency key only when nothing natural identifies the request. FIFO deduplication covers producer retries within five minutes and never covers the consumer, which always needs its own idempotency.

3.Never Assume Exactly-Once

Exactly-once delivery is not something a distributed system can offer, because a sender cannot tell a lost message from a lost acknowledgement. What can be built is exactly-once effect: at-least-once delivery plus a handler that produces the same result no matter how many times it runs. Every design that claims otherwise is relying on something being written down somewhere else.

Key features

A sender that receives no acknowledgement cannot distinguish a lost request from a lost response, which is why retries exist and why duplicates follow
SQS FIFO prevents a duplicate being added to the queue when `SendMessage` is retried within the 5-minute deduplication interval — a producer-side guarantee, not a processing one
Standard SQS queues make no deduplication guarantee at all
Lambda retries asynchronous invocations, and event source mappings redeliver batches, so a handler runs more than once as normal behaviour
S3 event notifications are delivered at least once, so an object-triggered pipeline sees repeats without anything having failed
AWS notes that requests can arrive late and from multiple actors, so a service must retain request history beyond the lifetime of the resource itself

What each AWS service actually promises

What each AWS service actually promises
Service	Delivery	Deduplication	What you still owe
SQS standard	At least once	None	An idempotent consumer
SQS FIFO	At least once to the consumer	Producer sends, within a 5-minute interval	An idempotent consumer, still
SNS	At least once	None	Idempotency in every subscriber
EventBridge	At least once	None	Idempotency in every target
S3 event notifications	At least once	None	Derive identity from the object key
Lambda, asynchronous	Retried by the service on error	None	An idempotent handler
Lambda event source mapping	Batches can be redelivered	None	Per-record idempotency, not per-batch
DynamoDB conditional write	Not applicable	Not applicable	Nothing — this is the mechanism that gives the guarantee

What each layer actually guarantees

Exactly-once delivery

Not available. A sender cannot tell a lost message from a lost acknowledgement, so it must retry, so duplicates exist.

At-least-once delivery

What SQS, SNS, EventBridge, S3 notifications and Lambda event sources actually provide. Assume it everywhere.

Bounded deduplication

FIFO removes duplicate sends within a 5-minute interval. Narrow, useful, and not a processing guarantee.

Exactly-once effect

What you build: a conditional write, a unique key, or stored durable state. This is the only layer under your control.

Signature

at-least-once delivery (theirs) + idempotent effect (yours) = exactly-once outcome

Delivery is at least once and you do not control it; effect is exactly once and you do. Every AWS guarantee — FIFO deduplication in particular — narrows one specific, documented window and leaves the consumer's obligation untouched. Write down what each guarantee does not cover, and put a conditional write there.