beginnerintermediateadvanced

AWS · Section 30

Step Functions — Workflow Orchestration

Level: intermediate
Read: 20 min
Concepts: 3

A Step Functions state machine defines a workflow as named states in JSON (Amazon States Language); a Task state runs one unit of work, and every execution's state transitions are recorded as execution history you can inspect without adding your own logging. Task, Parallel, and Map states carry Retry (re-run the same state with a growing wait, for transient errors) and Catch (reroute to a fallback state once retries are exhausted, for errors retrying will not fix) — two different tools for two different failure shapes. Parallel runs a fixed, hand-authored set of branches; Map runs one branch once per item of an input array, so its concurrency follows the data rather than the definition. TimeoutSeconds and the .waitForTaskToken callback pattern bound how long a Task can run or wait silently. Step Functions earns its place over orchestration logic hand-written inside one Lambda once that logic — retry loops, error branches, coordinating several services — has become the real complexity: a state machine makes it declarative and inspectable, and a Standard Workflow's one-year execution ceiling fits processes a single 15-minute Lambda invocation cannot.

This section

AWS Step Functions + Lambda Tutorial - Step by Step Guide in the Workflow StudioBe A Better Dev

What is true here

A state machine is JSON (ASL); a Task state runs one unit of work, and Step Functions records every state transition as execution history.
Retry re-runs the same state with a growing wait for transient errors (default 3 attempts, backoff ×2.0); Catch reroutes to a fallback state once retries are exhausted or the error is not retryable.
Parallel runs a fixed set of hand-authored branches; Map runs one branch once per array item, so its branch count follows the input data.
TimeoutSeconds/HeartbeatSeconds bound a Task's duration; .waitForTaskToken pauses a Task until an external system calls SendTaskSuccess or SendTaskFailure.
Move orchestration out of one Lambda into a state machine once retry logic and multi-service coordination are the real complexity — a single Lambda caps at a 15-minute execution timeout, a Standard Workflow at one year.

What you will be able to do

Read and write a basic state machine definition using Task states, Next/End, and StartAt
Choose Retry versus Catch for a given failure, and configure IntervalSeconds/MaxAttempts/BackoffRate correctly
Choose between a Parallel state and a Map state based on whether the branch count is fixed or data-driven
Recognize when orchestration logic hand-written inside one Lambda should move into a Step Functions state machine instead

From state machines and tasks to when Step Functions earns its place

State machines, tasks, executions

Retry, Catch, Parallel, Map, timeouts, callbacks

Step Functions vs. orchestration in one Lambda

Step Functions — Workflow Orchestration

State machines, tasks, and execution history; retries, catch handlers, parallel states, map patterns, timeouts, and callbacks; and when Step Functions earns its place over orchestration logic crammed into one Lambda.

State Machines, Tasks, and Executions

standardintermediate

A Step Functions workflow is called a state machine — a series of steps called states, defined in JSON (Amazon States Language). A Task state runs one unit of work, usually a Lambda function or another AWS service call. Each time you start the state machine, that run is called an execution, and Step Functions records every state transition as execution history you can inspect afterward.

Think of it as

Think of a state machine as a flowchart you hand to AWS instead of writing as if/else code. Each box is a state; a Task state is a box that does real work by calling a service. Every time the flowchart runs, that is one execution, and Step Functions keeps a step-by-step log of exactly which boxes ran, in what order, with what input and output — so a failure shows you precisely where it stopped rather than a stack trace you have to reconstruct yourself.

state-machine.jsonjson

{
  "StartAt": "ValidateOrder",
  "States": {
    "ValidateOrder": {
      "Type": "Task",
      "Resource": "arn:aws:lambda:us-east-1:123456789012:function:ValidateOrder",
      "Next": "ChargeCard"
    },
    "ChargeCard": {
      "Type": "Task",
      "Resource": "arn:aws:lambda:us-east-1:123456789012:function:ChargeCard",
      "End": true
    }
  }
}

What we're doing: See how a two-step order workflow reads as a state machine definition, and what its execution history shows after a run.

order-workflow.jsonjson

{
  "StartAt": "ValidateOrder",
  "States": {
    "ValidateOrder": {
      "Type": "Task",
      "Resource": "arn:aws:lambda:us-east-1:123456789012:function:ValidateOrder",
      "Next": "ChargeCard"
    },
    "ChargeCard": {
      "Type": "Task",
      "Resource": "arn:aws:lambda:us-east-1:123456789012:function:ChargeCard",
      "End": true
    }
  }
}

2: StartAt names the first state Step Functions enters when an execution begins.
6: Next chains ValidateOrder to ChargeCard — after ValidateOrder's Lambda returns, this is the state that runs next.
12: End: true marks ChargeCard as the workflow's terminal state; there is no further Next.

Why this works: Every state transition here — entering ValidateOrder, its Lambda's input/output, entering ChargeCard, the execution's final result — is recorded as execution history, so a failed order can be traced to the exact state and payload that caused it, without adding any logging code yourself.

One execution through a two-state workflow

StartExecution

ValidateOrder (Task)

ChargeCard (Task)

Execution result

Remember: A state machine is a JSON workflow of named states; a Task state does one unit of work (usually a Lambda call). Each run is an execution, and Step Functions records every state transition as execution history you can inspect without adding your own logging.

Retries, Catch, Parallel, Map, Timeouts, and Callbacks

coreintermediate

A Task, Parallel, or Map state can carry a Retry array (retry the same state on specific errors, with a growing wait between attempts) and a Catch array (route to a fallback state after retries are exhausted). A Parallel state runs a fixed set of branches at once; a Map state runs the same branch once per array item. TimeoutSeconds fails a Task that runs too long; the callback pattern (.waitForTaskToken) pauses a Task until an external system reports back.

Think of it as

Retry is "try this exact step again, waiting a bit longer each time" — for a transient glitch that often clears itself. Catch is "give up on this step and go run a different, known-good step instead" — for an error retrying will not fix. Parallel is a fixed number of different jobs starting at the same moment; Map is the same one job repeated once per item in a list, however long that list turns out to be.

What we're doing: See a Task state that retries a transient error, then falls back to a different state if retries are exhausted.

charge-card.jsonjson

"ChargeCard": {
  "Type": "Task",
  "Resource": "arn:aws:lambda:us-east-1:123456789012:function:ChargeCard",
  "Retry": [ {
    "ErrorEquals": ["States.TaskFailed"],
    "IntervalSeconds": 2,
    "MaxAttempts": 3,
    "BackoffRate": 2.0
  } ],
  "Catch": [ {
    "ErrorEquals": ["States.ALL"],
    "Next": "NotifyPaymentFailed"
  } ],
  "Next": "ShipOrder"
}

4: Retry only fires for States.TaskFailed — the wildcard that matches any Lambda-reported failure.
6: IntervalSeconds: 2, BackoffRate: 2.0 means the waits are 2s, then 4s, then 8s across the three attempts.
9: Catch only runs after Retry gives up (or for an error Retry does not match) — it is a separate, later check, not an alternative to Retry.
10: States.ALL as the sole entry catches whatever is left and routes to a human-facing fallback state instead of failing the whole execution.

Why this works: Retry and Catch are two different tools for two different failure shapes: Retry assumes the same call might succeed if tried again after a pause (a network blip); Catch assumes it will not, and the workflow needs a different path instead (charge declined) — using Catch alone for a transient error wastes the chance for an automatic recovery, and using Retry alone for a permanent error just delays the same failure three more times.

Retrying a Task with no MaxAttempts limit reasoning, or no Catch as a backstop

Wrong

json

"Retry": [ { "ErrorEquals": ["States.ALL"], "MaxAttempts": 3 } ]
// no Catch field at all

Better

json

"Retry": [ { "ErrorEquals": ["States.TaskFailed"], "MaxAttempts": 3 } ],
"Catch": [ { "ErrorEquals": ["States.ALL"], "Next": "NotifyPaymentFailed" } ]

What you see: A permanently failing Task (bad input, a bug, a declined card) retries three times, burning the backoff delay each time, and then the whole execution fails outright with no recovery state — the same outcome a Catch would have redirected cleanly.

Why: Retry alone has no fallback once MaxAttempts is exhausted — Step Functions defaults to failing the entire execution. A Catch is what gives a state machine a planned, inspectable failure path instead of an uncaught crash.

Parallel vs Map

Parallel state

+A fixed, hand-authored set of branches
+Each branch can do something different
+Branch count is part of the state machine definition

Map state

−One branch, run once per array item
−Every iteration runs the same steps
−Iteration count follows the input data at run time

Remember: Retry: same state, growing wait, for transient errors (default 3 attempts, backoff ×2). Catch: reroute to a different state once retries are exhausted or the error is not retryable. Parallel: a fixed set of different branches. Map: one branch, once per array item. TimeoutSeconds/HeartbeatSeconds bound how long a Task or a callback can run silently.

Step Functions vs. Orchestration Crammed Into One Lambda

standardintermediate

A single Lambda calling several other services in sequence, with its own hand-written retry loops and try/catch blocks, is orchestration logic hidden inside application code. Step Functions moves that same logic into a visible state machine definition, so each step, retry, and failure is inspectable in execution history instead of buried in a function's source and its logs.

Think of it as

One Lambda coordinating five other calls is a recipe written as a single paragraph — the steps are all there, but you have to read the whole function to find where it can fail. A state machine is the same recipe written as a numbered list with a name for each step, so a failure shows you exactly which numbered step it stopped on, without opening any code.

What we're doing: Compare the same order-fulfillment flow as hand-written orchestration inside one Lambda versus a Step Functions state machine.

orchestration-comparison.txttext

One Lambda: validateOrder(); chargeCard(); with try/catch and a manual
retry loop around chargeCard(); reserveInventory(); shipOrder() — all in
one function body, one execution timeout, one CloudWatch log stream

Step Functions: ValidateOrder -> ChargeCard (Retry + Catch) -> ReserveInventory
-> ShipOrder — each state visible and independently retryable, execution
history shows exactly which state ran with what input/output

1: Every retry, error branch, and step boundary lives inside the function's source code — reading the workflow means reading the whole function.
6: The same steps are now named states in a definition; a failure at ChargeCard shows up as that specific state's recorded input and output, not a line number in a stack trace.

Why this works: The two versions do the same work — the difference is whether the workflow is a first-class, inspectable thing (a state machine definition, execution history) or an implementation detail buried inside one function's control flow that only surfaces through logs you had to think to add.

Adding retry loops and cross-service orchestration to a Lambda as the workflow grows, instead of migrating to a state machine

Wrong

text

# Keep adding another try/catch block and another service call to the
# same Lambda every time the fulfillment flow grows a new step

Better

text

# Once the Lambda is coordinating more than a couple of services with its
# own retry/error logic, move that logic into a Step Functions state machine

What you see: The Lambda's execution time creeps toward the 15-minute function timeout as more steps get added, a failure three services deep only shows up as a single stack trace, and every new retry rule means editing and redeploying the function's code.

Why: A single Lambda has no built-in place to make a multi-step workflow visible — every additional step increases the function's own execution time against its 15-minute ceiling, and every additional retry/catch rule is more hand-written control flow to test and maintain, exactly the complexity Step Functions externalizes into a declarative, independently-inspectable definition.

Remember: Once retry logic, error branches, and multi-service coordination inside one Lambda are the actual complexity, that is a sign the workflow deserves to be a first-class Step Functions state machine — declarative, inspectable execution history — rather than more code buried in one function.