beginnerintermediateadvanced

System Design · Section 51

Batch and Stream Processing

Level: intermediate
Read: 20 min
Concepts: 3

Batch processing runs a job over a bounded, already-collected dataset on a schedule and then exits; stream processing runs continuously over an unbounded sequence of events with no defined end — the choice trades result freshness against operational simplicity. A stream job needs its own vocabulary because its input never stops: windowing (tumbling, sliding, or session) bounds it into computable slices, state holds the running aggregate a window needs, checkpointing periodically saves that state together with the input offset, and replay uses a checkpoint to resume from that point rather than reprocessing everything from the start. Throughput and latency trade off directly through batching — bigger batches raise throughput and raise per-event latency together — and backpressure is what a stream processor faces once input arrives faster than it can keep up, a concern introduced here and covered in full next.

This section

Batch Processing vs Stream Processing | System Design Primer | Tech PrimersTech Primers

What is true here

Batch: bounded input, scheduled, exits when done. Stream: unbounded input, runs continuously, never exits on its own.
Windowing (tumbling, sliding, session) is what makes an aggregate computable over a stream that never ends.
Checkpointing must save state and input offset together — either alone leaves recovery either data-losing or position-wrong.
Batching trades latency for throughput; backpressure is the failure mode once input outruns the processing ceiling either way.

What you will be able to do

Choose batch or stream processing based on the actual freshness requirement, not which sounds more sophisticated
Pick the right window type (tumbling, sliding, session) for a given aggregation need
Explain why a checkpoint must save state and offset together, and size a checkpoint interval against replay cost
Recognize the throughput/latency trade-off batching creates, and identify backpressure as its downstream concern

Two processing paradigms

Batch and stream processing as genuinely different answers to different freshness requirements, not a maturity ladder.

Batch vs stream processing

coreintermediate

Batch processing runs a job over a bounded, already-collected set of data — yesterday's orders, a full table export — on a schedule or on demand, then finishes. Stream processing runs continuously over an unbounded sequence of events as they arrive, one at a time or in small groups, and never "finishes" in the same sense. The choice is about how fresh the result needs to be versus how much complexity you are willing to run permanently.

Think of it as

Batch is doing a week's laundry every Sunday: you wait until you have a full load, run it once, and it is done until next Sunday. Stream is washing each item the moment it gets dirty: the washing machine runs continuously, output is available almost immediately, but you now have a machine running all week instead of an hour on Sunday, and you have to handle one dirty sock arriving mid-cycle instead of waiting for a full load.

text

// Batch: bounded input, runs once, exits
job.read(source="orders_2026_08_24.parquet")
   .aggregate(by="customer_id")
   .write(sink="daily_totals")
// process exits when the file is fully read

// Stream: unbounded input, runs forever
stream.read(source="orders-topic")
      .aggregate(by="customer_id", window="1h")
      .write(sink="running_totals")
// process keeps running as new events arrive

What we're doing: Compare how the same "revenue per customer" report is produced by a batch job and a stream job.

batch-vs-stream-revenue.txttext

BATCH (runs at 02:00 daily via cron):
1. Read every order row from yesterday's partition
   of the orders table (a bounded, closed dataset --
   yesterday will never gain a new row).
2. Group by customer_id, sum order_total.
3. Write the result to daily_revenue_by_customer.
4. Job exits. Total runtime: 12 minutes.
   Report is 2-26 hours stale, depending on when in
   the day the order happened.

STREAM (deployed once, runs continuously):
1. Consume the orders-topic as new orders are
   published, one event at a time.
2. Maintain a running per-customer total in state,
   updated on every new order event.
3. Emit the updated total to a live dashboard within
   ~1 second of the order event arriving.
4. Job never exits -- it is redeployed, not rerun,
   when the code changes.

2: Yesterday's partition is bounded by definition -- no new order can ever be added to it, which is what makes the batch job's output reproducible on rerun.
12: State here means the running total the stream job keeps in memory (or a local store) between events -- without it, each event would need to re-read every prior order to compute a new sum.
15: A stream job is redeployed on a code change, not "rerun" the way a batch job is -- it has no natural end to rerun from.

Why this works: The same business question — revenue per customer — produces two different systems depending on whether the answer can wait until tomorrow (batch, bounded, simple) or needs to be current within a second (stream, unbounded, continuously running).

Building a stream pipeline for a report that only needs to be accurate once a day

Wrong

text

// Continuous stream job, checkpointing,
// state store, and on-call rotation --
// for a report only read once, at 9am

Better

text

// Nightly batch job over yesterday's bounded
// partition -- one cron trigger, no standing
// infrastructure, reruns cleanly on failure

What you see: The team ends up operating a continuously-running stream job — with its own state store, checkpointing, and on-call burden — for a report nobody looks at more than once a day, and every stream-specific failure mode (state growth, checkpoint lag, rebalance stalls) now has to be diagnosed for a freshness requirement the batch job would have met trivially.

Why: Stream processing's cost is standing infrastructure that runs whether or not anyone is watching; that cost buys freshness. When the actual requirement tolerates hours of staleness, a scheduled batch job gets the same answer for a fraction of the operational surface — matching the tool to the actual latency requirement, not the more sophisticated one.

Bounded job vs continuous job

Batch

+Input is a fixed file or table snapshot
+Job starts, processes everything, exits
+Rerunning the same input reproduces the same result

Stream

−Input is an unbounded sequence of events
−Job starts once and keeps running indefinitely
−Recovery resumes from a checkpoint, not a full rerun

Batch vs stream processing — the decision-relevant differences

Batch vs stream processing — the decision-relevant differences
Dimension	Batch processing	Stream processing
Input	Bounded — a fixed dataset with a known end	Unbounded — events keep arriving indefinitely
Trigger	Schedule (cron, nightly) or on-demand job	Continuous — the job runs and never exits
Latency to result	Minutes to hours (matches the schedule)	Sub-second to a few seconds
Failure recovery	Rerun the whole job over the same bounded input	Resume from a checkpoint — rerunning from scratch is often infeasible
Typical use	Billing runs, data warehouse ETL, model training	Fraud detection, live dashboards, alerting

Remember: Batch processes a bounded, already-collected dataset on a schedule and then exits; stream processes an unbounded sequence of events continuously and never exits. Choose based on whether the result needs to be current within seconds (stream) or can wait for the next scheduled run (batch) — not based on which sounds more modern.

The vocabulary a never-ending stream needs

Windowing, state, checkpointing and replay, plus the throughput/latency trade-off and the backpressure concern it creates.

Windowing, state, checkpointing and replay

coreintermediate

A stream never ends, so "sum all the orders" needs a boundary — a window is that boundary, a slice of time (or count) events get grouped into before an aggregate is computed. State is what a stream job remembers between events, such as a running total per customer. Checkpointing periodically saves that state (and how far into the input the job has read) so a crash does not lose it. Replay re-reads events from an earlier point, using a checkpoint to pick up exactly where processing left off instead of starting over.

Think of it as

Think of a stream job like a cashier at a 24-hour store who never gets to close the register and total the day. A window is the cashier deciding to total up "every sale in this 15-minute block" instead of waiting for a day that never ends. State is the running subtotal the cashier keeps on a notepad between sales. Checkpointing is photographing that notepad every few minutes and filing the photo safely. Replay is a new cashier taking over after a fire: instead of guessing, they pull the last photo and re-ring only the sales that happened since it was taken.

text

// Tumbling window: 15-minute buckets, no overlap
stream.window(type="tumbling", size="15m")
      .aggregate(sum("order_total"), by="region")

// Sliding window: 5-minute window, advances every 30s
stream.window(type="sliding", size="5m", slide="30s")
      .aggregate(avg("latency_ms"))

// Session window: closes after 30 min of no events
stream.window(type="session", gap="30m")
      .aggregate(count(), by="user_id")

// Checkpoint every 10s: saves state + input offset
stream.checkpoint(interval="10s", store="s3://ckpts/")

What we're doing: Trace a per-region order-count stream job through a crash, checkpoint restore, and replay.

windowing-checkpoint-replay.txttext

1. Job starts consuming orders-topic at offset 0,
   using a 15-minute tumbling window keyed by region.
2. t=00:00-00:15 window: state = {us: 120, eu: 84}
   (running counts, updated as each order event
   arrives and increments its region's counter).
3. t=00:10: job checkpoints. Saved: state snapshot
   {us: 74, eu: 51} and input offset 18,402.
4. t=00:14: job process crashes (OOM). All in-memory
   state since the checkpoint is lost.
5. t=00:14: orchestrator restarts the job. It loads
   the checkpoint: state = {us: 74, eu: 51},
   offset = 18,402.
6. Job replays events from offset 18,402 onward --
   re-consuming the events between the checkpoint
   and the crash, replaying them into state.
7. t=00:15: window closes with state = {us: 120,
   eu: 84} -- identical to step 2, because replay
   reprocessed exactly the events the checkpoint
   had not yet captured.

3: The window boundary (00:00-00:15) is what turns an endless stream into a bounded question -- "how many orders in this block" -- the same kind of question a batch job answers over a bounded file.
8: The checkpoint pairs state with an input offset -- saving one without the other would leave the job unable to know which events already contributed to the saved totals.
17: Replay re-consumes events starting exactly at the checkpointed offset, not from the beginning of the stream -- that is what keeps recovery fast on a stream that may have billions of prior events.

Why this works: The step-5 restart shows why checkpointing and replay are paired concepts: a checkpoint alone (state saved, but no record of position) cannot resume correctly, and replay alone (position saved, but no state saved) would resume position-correct but with counts silently reset to zero.

Checkpointing the input offset without checkpointing the accumulated state

Wrong

text

// Checkpoint only saves "we've read up to
// offset 18,402" -- not the running counts
checkpoint.save(offset=18402)
// on restart: resumes reading at 18402,
// but state = {} (reset to empty)

Better

text

// Checkpoint saves BOTH the offset and the
// state snapshot together, atomically
checkpoint.save(offset=18402, state={
    "us": 74, "eu": 51
})
// on restart: resumes reading at 18402
// AND restores state to {us: 74, eu: 51}

What you see: After every crash-and-restart, window totals are silently too low — the job correctly avoids reprocessing already-read events (no duplicates), but it also does not recover the counts those events had already contributed, so a window that should read {us: 120, eu: 84} at close instead reads {us: 46, eu: 33}, the count of only the events read after the restart.

Why: Offset and state answer two different questions — "which events have I read" and "what have those events added up to" — and a correct resume needs both answered consistently. Saving only the offset avoids reprocessing, which looks like success, but it silently discards the very thing windowing exists to compute.

From raw events to a recovered, checkpointed aggregate

Unbounded events

Windowing

groups events into a bounded slice

State

running aggregate per key

Checkpoint

saves state + offset periodically

Replay on restart

resumes from last checkpoint, not from zero

The three window types and when each fits

The three window types and when each fits
Window type	Boundary rule	Example use
Tumbling	Fixed size, no overlap — each event in exactly one window	Orders per 15-minute block for a dashboard
Sliding	Fixed size, overlapping — one event can be in several windows	A trailing 5-minute error rate, recomputed every 10 seconds
Session	Closes after a gap of inactivity — variable length per key	A user's browsing session, closed after 30 minutes idle

Remember: Windowing turns an endless stream into bounded, computable slices — tumbling (fixed, no overlap), sliding (fixed, overlapping), or session (closes on a gap). State is the running aggregate a window needs; checkpointing saves that state together with the input offset; replay uses a checkpoint to resume from that point instead of from zero. Save state and offset together, or recovery silently under-counts.

Throughput, latency and backpressure in stream processing

standardintermediate

Throughput is how many events a stream job processes per second; latency is how long one event takes to go from arrival to processed result. Batching events together raises throughput (fewer, larger operations) but raises latency (an event waits for its batch to fill). Backpressure is what happens when events arrive faster than the job can process them — a concern every stream processor must handle, since input never pauses to let it catch up on its own.

Think of it as

A single-lane toll booth shows the trade-off directly. Processing cars one at a time, instantly, gives the lowest latency per car but caps throughput at whatever one booth can handle. Grouping cars into a shuttle that waits until it is full raises throughput per trip but makes the first car in the shuttle wait for every other seat to fill — higher latency for that car. Backpressure is the traffic jam that forms when cars arrive faster than the booth (or the shuttle schedule) can absorb them; something upstream has to slow down, or the queue grows without bound.

text

// Per-event processing: lowest latency, lower throughput
stream.process(event -> handle(event))

// Micro-batching: higher throughput, added latency
stream.batch(size=500, maxWait="50ms")
      .process(batch -> handleBatch(batch))
      // each event now waits up to 50ms for its
      // batch to fill (or the wait to time out)

// Backpressure signal (consumer tells producer to slow down)
if queue.size() > HIGH_WATERMARK:
    consumer.pause()   // stop pulling more events
    // producer/broker now buffers instead of this job

What we're doing: Show the same event stream under per-event processing and micro-batching, and where backpressure appears if the consumer falls behind.

throughput-latency-tradeoff.txttext

Per-event mode:
  event arrives -> processed immediately -> result out
  Latency per event: ~2ms
  Sustained throughput ceiling: ~5,000 events/sec
  (limited by fixed per-event overhead: 2ms floor)

Micro-batch mode (batch size 500, max wait 50ms):
  events buffer -> batch fills (or 50ms elapses)
  -> whole batch processed -> results out together
  Latency per event: up to 50ms (waits for the batch)
  Sustained throughput: ~50,000 events/sec
  (per-event overhead paid once per 500 events)

If events arrive at 80,000/sec (above either mode's
ceiling): the input queue grows every second instead
of draining -- this is backpressure. Something must
either slow the producer down or the queue grows
without bound until the job runs out of memory.

8: Batching amortizes fixed per-event overhead (connection handling, serialization) across many events, which is what raises the throughput ceiling from 5,000 to 50,000 events/sec here.
10: The added latency is bounded by the max wait, not the batch size alone -- an event that arrives right as a batch fills waits far less than one that arrives right after a batch just closed.
16: This is backpressure: input exceeding either mode's throughput ceiling, so the unprocessed queue grows every second rather than draining -- the full set of ways to handle it is its own topic, covered next.

Why this works: The same stream shows both halves of the trade-off at once: switching from per-event to micro-batching moves throughput and latency in opposite directions, and neither mode is exempt from backpressure once input exceeds its ceiling — batching raises the ceiling, it does not remove it.

Per-event processing vs micro-batching

Per-event

+Each event processed as soon as it arrives
+Lowest possible latency per event
+Per-event overhead paid on every single event — lower ceiling on throughput

Micro-batching

−Events buffered briefly, processed as a group
−Higher throughput — overhead amortized across the batch
−Added latency: an event waits for its batch to fill or time out

Increasing micro-batch size to fix a throughput problem without checking the resulting per-event latency

Wrong

text

// Throughput too low at batch size 500 --
// "just" raise the batch size to 20,000
stream.batch(size=20000, maxWait="2s")

Better

text

// Check the latency budget first: if consumers
// need results within 200ms, a 2s max wait
// already violates it regardless of throughput
stream.batch(size=2000, maxWait="150ms")
// raises throughput within the actual latency
// budget, instead of past it

What you see: Throughput improves as expected, but a downstream alerting system that depended on results within a few hundred milliseconds starts missing its own SLA — events are now correctly processed, just too late to matter, and the regression shows up as a latency-sensitive consumer breaking, not as an error in the stream job itself.

Why: Batch size and max-wait set an upper bound on added latency directly — a larger batch (or longer max wait) always raises the ceiling on how long an event can wait for its batch to fill. Tuning purely for throughput without checking that ceiling against the actual latency requirement trades away a constraint nobody meant to give up.

Remember: Throughput (events/sec) and latency (time per event) trade off through batching — bigger batches raise throughput and raise per-event latency at the same time. Backpressure is what happens when input arrives faster than either mode's ceiling; a stream processor must handle it because, unlike a batch job's fixed file, the input never pauses on its own.