Filter concepts by levelShowing all levels.

AWS · Section 6

AWS CLI, SDKs, and APIs

Level
intermediate
Read
24 min
Concepts
4

The Console, CLI, and SDKs are three interfaces onto the same underlying AWS API — the same quotas, error codes, and request signing apply no matter which one is used. This section covers CLI profile/region/JMESPath basics, what an SDK client does automatically (signing, retries, pagination, waiters), how to read a real API error and check a service quota instead of guessing from the console, and why repeatable tasks should be automated rather than clicked through by hand.

What is true here

  1. --profile selects credentials and default region; --query (JMESPath) reshapes a response client-side — safer to script against than text-output column position.
  2. SDK clients sign every request, retry transient/throttling errors with backoff, and (via a paginator) page through long list results automatically.
  3. An API error's Code and Message identify the real cause — a quota limit and a permissions denial can look identical in a summarized console banner.
  4. Service quotas are scoped per account, per Region — a limit increase in one Region does not carry over to another.
  5. A one-off action is fine in the console; anything repeated belongs in a script, CLI command, or IaC template instead.

What you will be able to do

  • Use --profile, --region, and --query together to script against a specific account, region, and response shape
  • Explain what an SDK client already handles automatically — signing, retry backoff, pagination — before writing a manual loop
  • Read an API error's Code and Message to tell a quota problem from a permissions problem
  • Check and, if needed, request an increase for a specific service quota in the correct Region
  • Decide when a task belongs in the console versus when it should be automated
Three doors, one underlying API

Console

good for exploration, one-off actions

CLI

profiles, regions, --query

SDK

retries, pagination, waiters built in

Same API underneath

same quotas, same errors, same signing

  1. Console — good for exploration, one-off actions
  2. CLI — profiles, regions, --query
  3. SDK — retries, pagination, waiters built in
  4. Same API underneath — same quotas, same errors, same signing

CLI, SDKs, and APIs

CLI basics, what an SDK client does automatically, reading real API errors and quotas, and when to automate.

AWS CLI basics: profiles, regions, and JMESPath queries

coreintermediate

The AWS CLI wraps every service API as a command. A named profile picks which credentials and region to use, and `--query` filters a large JSON response down to just the fields you need with JMESPath.

Think of it as

A universal remote with one button per API call, and a profile switch on the side that decides which account and region the button presses land in.

text
aws <service> <operation> [--profile name] [--region region] [--query jmespath] [--output format]

What we're doing: List running instance IDs only, instead of the full describe-instances response.

terminalbash
aws ec2 describe-instances \
  --query "Reservations[].Instances[?State.Name=='running'].InstanceId[]" \
  --output text
2
The JMESPath filter [?State.Name=='running'] runs client-side, after AWS returns every instance regardless of state.

Why this works: describe-instances returns a deeply nested structure — Reservations containing Instances — that is rarely useful to read as-is. --query reshapes it into exactly the list a script needs, without a separate parsing step.

Assuming --region on one command changes the default for later commands

Wrong

bash
aws ec2 describe-instances --region us-west-2
aws ec2 describe-instances   # still queries the profile's default region, not us-west-2

Better

bash
export AWS_DEFAULT_REGION=us-west-2   # or use --profile with region set in ~/.aws/config
aws ec2 describe-instances
aws ec2 describe-instances

What you see: A second command in the same session returns instances from the wrong region, with no error to indicate why.

Why: --region is a per-invocation flag, not session state — the CLI has no memory between commands. Only an environment variable, a profile's configured region, or the flag itself (repeated) determines each call's region.

One CLI call, four independent flags

aws ec2 describe-instances \ --profile staging --region eu-west-1 \ --query "Reservations[].Instances[].InstanceId" --output text

--profile staging

profile — which credentials and default settings to use

--region eu-west-1

region — overrides the profile's default region, this call only

--query "Reservations[].Instances[].InstanceId"

query — JMESPath — reshapes the response client-side

--output text

output — the response format

  • Whole: aws ec2 describe-instances \ --profile staging --region eu-west-1 \ --query "Reservations[].Instances[].InstanceId" --output text
  • --profile staging — profile: which credentials and default settings to use
  • --region eu-west-1 — region: overrides the profile's default region, this call only
  • --query "Reservations[].Instances[].InstanceId" — query: JMESPath — reshapes the response client-side
  • --output text — output: the response format

Common AWS CLI flags

Common AWS CLI flags
FlagPurpose
--profile <name>Use a named profile's credentials and settings
--region <region>Override the region for this call only
--output json|yaml|text|tableResponse format
--query <jmespath>Filter/reshape the response before printing
--dry-runValidate permissions and parameters without executing

Together

bash
aws ec2 describe-instances \
  --profile staging --region eu-west-1 \
  --query "Reservations[].Instances[].InstanceId" --output text

Remember: --profile picks credentials and default region; --query (JMESPath) reshapes the response client-side — always safer to script against than text-output column position.

See also: sdk clients and resilience

SDK clients: retries, paginators, and waiters

coreintermediate

An AWS SDK client does three things behind the scenes on every call: signs the request with your credentials, retries it automatically on throttling or transient failure, and — for calls returning long lists — a paginator saves you from writing your own next-page loop.

Think of it as

A postal courier who not only delivers the letter but also stamps it correctly, redelivers it automatically if the recipient was briefly unavailable, and keeps making trips until every box in a large shipment has arrived.

python
paginator = client.get_paginator("list_objects_v2")
for page in paginator.paginate(Bucket="my-bucket"):
    ...

What we're doing: Compare hand-rolled pagination against a paginator for the same list operation.

list_all_objects.pypython
# manual pagination
token = None
keys = []
while True:
    kwargs = {"Bucket": "my-bucket"}
    if token:
        kwargs["ContinuationToken"] = token
    resp = client.list_objects_v2(**kwargs)
    keys += [o["Key"] for o in resp.get("Contents", [])]
    if not resp.get("IsTruncated"):
        break
    token = resp["NextContinuationToken"]

# with a paginator
paginator = client.get_paginator("list_objects_v2")
keys = [o["Key"] for page in paginator.paginate(Bucket="my-bucket") for o in page.get("Contents", [])]
6
ContinuationToken must be tracked and conditionally passed by hand — easy to get wrong on the first page.
8
IsTruncated has to be checked every loop, or the last page silently gets dropped or the loop never ends.

Why this works: Every list-style AWS API paginates the same way in principle but with slightly different field names per service — a paginator hides that inconsistency instead of every caller re-deriving it.

Reading only the first page of a list-style API and assuming it is complete

Wrong

python
resp = client.list_objects_v2(Bucket="my-bucket")
keys = [o["Key"] for o in resp["Contents"]]   # only up to 1,000 keys

Better

python
paginator = client.get_paginator("list_objects_v2")
keys = [o["Key"] for page in paginator.paginate(Bucket="my-bucket") for o in page.get("Contents", [])]

What you see: A script that worked in testing (small bucket) silently processes only the first 1,000 objects once the bucket grows past a single page.

Why: list_objects_v2 caps each response at 1,000 keys and sets IsTruncated when more exist — a call that never checks for a next page has no signal that anything was missed, it just returns fewer results than reality.

What a client call does before your code sees a result

Sign

SigV4, using resolved credentials

Send

to the resolved region

Retry

exponential backoff on throttling

Paginate

if the operation returns pages

  1. Sign — SigV4, using resolved credentials
  2. Send — to the resolved region
  3. Retry — exponential backoff on throttling
  4. Paginate — if the operation returns pages

Remember: SDK clients already sign, retry with backoff, and (via a paginator) page through long list results — reach for the paginator/waiter before hand-rolling the same loop.

See also: cli basics · inspecting apis and quotas

Inspecting APIs, quotas, and error messages

standardintermediate

Every AWS Console click maps to an underlying API call, and every account has numeric limits — service quotas — on how many of a resource it can create. Reading the actual API error and checking the quota beats guessing from the console alone.

Think of it as

The console is a GUI on top of the same API you can call directly — like a car's dashboard sitting on top of the engine. When something goes wrong, looking at the engine (the API error) tells you more than staring at the dashboard light.

bash
aws service-quotas get-service-quota --service-code ec2 --quota-code L-1216C47A

What we're doing: Distinguish a quota error from a permissions error by reading the actual error code, not just "it failed."

error-response.jsonjson
{
  "Error": {
    "Code": "VcpuLimitExceeded",
    "Message": "You have requested more vCPU capacity than your current vCPU limit of 32 allows..."
  }
}
2
The error Code identifies exactly what kind of failure this is — here, a quota, not a permissions or validation problem.
3
The Message states the current limit explicitly (32 vCPUs) — enough to file a targeted Service Quotas increase request.

Why this works: A quota error and an IAM permissions error can both surface in the console as a generic red banner — the underlying API error Code and Message name the actual cause precisely, and quota errors specifically point at Service Quotas, not IAM.

Requesting a quota increase in the wrong Region

Wrong

bash
aws service-quotas request-service-quota-increase --region us-east-1 \
  --service-code ec2 --quota-code L-1216C47A --desired-value 64
# but the workload actually runs in eu-west-1

Better

bash
aws service-quotas request-service-quota-increase --region eu-west-1 \
  --service-code ec2 --quota-code L-1216C47A --desired-value 64

What you see: The increase is approved, but launches in the actual working Region still fail with the same VcpuLimitExceeded error.

Why: Most service quotas are scoped per Region, not per account globally — raising a limit in one Region has no effect on any other Region's independent limit.

Remember: Read the API error Code and Message directly rather than a summarized console banner, and remember most quotas are per-Region, not account-wide.

See also: sdk clients and resilience · pricing dimensions

Automation first for repeatable tasks

standardintermediate

A console click that is never repeated is fine as a console click. A task done more than once — creating an environment, rotating a resource, running a routine check — should be a script or IaC template instead, so it runs the same way every time.

Think of it as

A recipe written down once versus cooking from memory each time — the written recipe produces the same dish whether you or someone else follows it a year later.

text
Console click  →  did it once, correctly, this time
Script / CLI / IaC  →  does it the same way, every time, reviewably

What we're doing: Compare a one-off console action against the same action automated, and see what the automated version gains.

create-bucket.shbash
aws s3api create-bucket --bucket my-app-uploads --region eu-west-1 \
  --create-bucket-configuration LocationConstraint=eu-west-1
aws s3api put-bucket-encryption --bucket my-app-uploads \
  --server-side-encryption-configuration file://sse-config.json
aws s3api put-public-access-block --bucket my-app-uploads \
  --public-access-block-configuration BlockPublicAcls=true,IgnorePublicAcls=true,BlockPublicPolicy=true,RestrictPublicBuckets=true
1
Every setting this bucket needs is explicit in the script — nothing depends on someone remembering to also check three boxes in the console.
3
Encryption and public-access blocking are guaranteed identical the next time this script runs, for a second bucket, a year later.

Why this works: A script committed to source control is reviewable before it runs and reproducible after — a sequence of console clicks is neither, and is easy to do slightly differently the second time without noticing.

Fixing a production issue by hand in the console "just this once"

Wrong

text
# Console: manually bump an Auto Scaling Group's max size during an incident

Better

text
# Update the value in the IaC template and apply it — even under time
# pressure, so the change is captured rather than silently drifting

What you see: Months later, nobody remembers why the deployed max size does not match what the IaC template says, and the next `terraform apply` silently reverts the change.

Why: A console fix resolves the immediate incident but leaves infrastructure code describing a different reality than what is actually running — the next automated deployment overwrites the manual fix without anyone deciding that should happen.

Remember: A one-off console action is fine; anything repeated becomes a script, CLI command, or IaC template — reviewable before it runs, reproducible after.

See also: cli basics · sdk clients and resilience

Advertisement