SNS and Event Fan-Out

Topics, subscriptions, and filtering; the SNS + SQS fan-out pattern; and when pub/sub beats a direct call

One publish, many subscriberseach subscription gets its own full copy
Filter policy per subscriptionnot per topic — each subscriber opts into a subset
One SQS queue per consumera shared queue splits messages instead of duplicating them
Publisher never blocks on a subscriberpub/sub decouples both ends

A topic is a named channel a publisher sends messages to, without knowing who — if anyone — is listening. A subscription attaches one delivery target (SQS, Lambda, HTTP(S), email, SMS, mobile push) to a topic; every subscription gets its own copy of each message. A filter policy on a subscription drops messages that do not match, so one topic can serve subscribers that each want a different subset. Fan-out is what happens when a topic has multiple subscriptions: one publish reaches every matching subscriber in parallel.

Key features

Delivery targets: Amazon SQS, AWS Lambda, HTTP(S) endpoints, email, SMS, mobile push, Amazon Data Firehose, and select third-party service providers
A filter policy is JSON attached to a subscription; without one, that subscriber receives every message published to the topic
Filter policies match on message attributes or the message body (body matching requires the payload to be well-formed JSON)
Server-side delivery failures retry on a per-protocol schedule: SQS/Lambda get up to 100,015 attempts over 23 days; SMTP/SMS/mobile push get up to 50 attempts over 6 hours
Client-side errors (e.g. the subscribed endpoint was deleted) are not retried at all — SNS discards the message immediately unless a dead-letter queue is attached
A dead-letter queue is attached per subscription, not per topic — because delivery, and delivery failure, happens at the subscription level

One publish, three independent deliveries

Publish(message)matchesfiltermatchesfilterno filter —gets everything

Publisher

SNS topic

SQS subscriber

Lambda subscriber

Email subscriber

Signature

Topic = named channel; subscription = one delivery target attached to it, each with its own filter policy and retry schedule. Fan-out is one publish reaching every matching subscriber in parallel. Client-side errors are not retried; server-side errors retry per-protocol (SQS/Lambda: ~23 days; SMTP/SMS/push: ~6 hours) before being discarded — attach a DLQ per subscription to catch what would otherwise be lost.

SNS pushes; SQS holds. Subscribe several SQS queues to one SNS topic, and every message published reaches every queue as its own durable copy, each consumer polling and processing at its own pace. If one consumer's queue backs up or its worker crashes, the message still sits safely in that queue — the other consumers, and the publisher, are unaffected.

Key features

Each SQS queue subscribed to an SNS topic gets its own full copy of every message published — this is what makes it fan-out rather than a single shared queue
A consumer with SQS in front of it can poll at its own rate, batch, and retry from the queue — SNS alone offers no polling model and no consumer-controlled retry
Decoupling holds at both ends: the publisher never references a consumer, and consumers never reference each other or the publisher
A queue subscribed to a topic must have an access policy granting the SNS service principal permission to send messages to it
A new subscriber is added by subscribing a new queue to the existing topic — no change to the publisher or to any other consumer
The delivered SQS message wraps the original publish in a JSON envelope carrying Message, MessageId, TopicArn, Timestamp, and (for HTTPS-confirmable protocols) an UnsubscribeURL

SNS alone vs SNS + SQS per consumer

SNS alone (push only)

A down or slow consumer relies entirely on SNS's own retry schedule
No consumer-side polling, batching, or backpressure control
A client-side failure (e.g. deleted endpoint) is not retried at all

SNS + SQS per consumer

Each queue durably holds messages until its consumer polls them
Consumer controls its own poll rate, batch size, and retry/backoff
One consumer falling behind or crashing does not affect the others

Signature

SNS topic --subscribe--> SQS queue (× N consumers), each queue polled independently

SNS fans one publish out to N subscribers; putting an SQS queue in front of each consumer turns "push and hope" into "push into a durable, pollable buffer per consumer." One queue per consumer for true fan-out — a shared queue splits messages instead of duplicating them. The queue's access policy must allow the SNS service principal to send to it.

A direct synchronous call means Service A calls Service B and waits for a response — A cannot finish its own work until B answers, and A must know B's address. Publishing to an SNS topic means A hands off a message and moves on immediately — A never learns how many services end up handling it, or how long they take. Pub/sub fits when A does not need an answer to continue, and when more than one downstream service should react to the same event.

Key features

A direct synchronous call blocks the caller until the callee responds — if the callee is unavailable, the caller's own operation fails or stalls
Publishing to an SNS topic returns as soon as SNS accepts the message — the publisher does not wait on any subscriber, and a subscriber being down does not fail the publish
Pub/sub decouples the publisher from the very existence of its subscribers: adding a new subscriber to a topic requires no change to the publisher
The SNS fan-out scenario — one published event reaching multiple independent subscribers in parallel — is the documented case for choosing pub/sub over a synchronous call: e.g. one order-placed event reaching a fulfillment queue and a separate analytics queue at once
A direct call fits when the caller genuinely needs the result before it can proceed (e.g. an authorization check) — pub/sub does not remove that requirement, it only fits when the caller does not need one

Command

sync-vs-publish.pypython

# Direct synchronous call — order_service blocks until payment_service responds
response = payment_client.charge(order_id, amount)   # order_service waits here

# Pub/sub — order_service hands off and continues immediately
sns.publish(TopicArn=ORDER_PLACED_TOPIC, Message=json.dumps(order))
# fulfillment and analytics each subscribe independently; order_service
# never learns how many subscribers exist or when they finish

Choose a direct synchronous call when the caller needs the result to proceed (e.g. an authorization decision). Choose pub/sub (SNS) when the caller does not need a response, or when more than one downstream service should react to the same event without the publisher knowing who they are.

Spotted a mistake on this page? Tell us.

Read Sheet

1.SNS Topics, Subscriptions, Filtering, and Fan-Out

2.SNS + SQS Fan-Out for Decoupling

3.Pub/Sub vs Direct Synchronous Calls