AWS Interview-Level Questions

# The follow-up on evaluation order, answered properly.

Q: "You attached a policy allowing s3:GetObject and the call
    still returns AccessDenied. Where do you look, in order?"

1. An explicit Deny anywhere. Identity policy, resource policy,
   VPC endpoint policy, or a service control policy on the OU.
   A Deny cannot be overridden by any Allow.
2. A service control policy or permission boundary that does not
   allow the action. Neither of these grants anything; they cap
   what the identity policy is able to grant.
3. The resource policy, if the caller is in another account.
   Cross-account access needs an allow on both sides.
4. A condition that is not satisfied — aws:SourceVpce,
   aws:PrincipalOrgID, or an encryption-context condition on the
   KMS key the object uses.
5. The KMS key policy itself, for an encrypted object. The
   s3:GetObject allow is worthless without kms:Decrypt.

Point 5 is the one most candidates never reach, and it is the
most common real cause.

# The NACL follow-up, answered.

Q: "The security group allows 443 inbound. The request goes out
    and the response never comes back. What is it?"

A: The NACL. A security group is stateful — it remembers the
   outbound connection and lets the reply back in automatically.
   A NACL does not remember anything, so the reply is evaluated
   as a brand new inbound packet, on an ephemeral source port.

   The reply to an outbound request arrives on a port in the
   ephemeral range, so the NACL needs:

     inbound  ALLOW  TCP  1024-65535  from 0.0.0.0/0

   Without that rule the outbound request succeeds and the
   response is dropped at the subnet edge, which reads exactly
   like a timeout in the application.

Q: "So when do you use a NACL at all?"
A: When you need a deny. Security groups cannot express one, so
   blocking a specific CIDR at the subnet edge is the case NACLs
   are actually for.

# The throttling follow-up, answered by invocation type.

Synchronous (API Gateway, an SDK Invoke call)
  → the caller receives 429 TooManyRequestsException immediately.
  → the user sees an error, so this is the loud case.

Asynchronous (S3 event, SNS, EventBridge)
  → Lambda queues the event and retries, by default twice, with
    a delay. Nothing is lost quickly and nothing surfaces either.
  → the symptom is latency, not errors. Set a destination or a
    dead-letter queue, or the failure is silent after the retries.

Event source mapping (SQS, Kinesis, DynamoDB Streams)
  → the poller backs off and retries. For SQS the message
    returns to the queue after its visibility timeout.
  → the symptom is a growing queue depth, which is why queue
    depth is the alarm that catches this and error rate is not.

# The mistake this catches: "we would see the errors". You would
# see them for one of the three, and the other two are the ones
# that page you at 3 a.m. as a backlog.

# The follow-ups, answered.

"Can a read replica be your HA story?"
  No. Replication is asynchronous, so promoting one can lose the
  writes that had not replicated yet, and promotion is a manual
  or scripted action rather than an automatic failover. It is a
  reasonable disaster-recovery component and it is not Multi-AZ.

"When would you deliberately not use Aurora?"
  When you need an extension or engine behaviour Aurora does not
  carry; when the workload is small enough that the smallest
  sensible Aurora footprint costs more than the RDS instance it
  replaces; or when you want a plain engine you can move off AWS
  without a migration.

"Product asks for a new report next week."
  Relational: write the query, add an index if it is slow.
  DynamoDB: if the pattern is not covered by the key or an
  existing index, you add a GSI — or you export to S3 and query
  it elsewhere. That is the flexibility you traded for the
  latency guarantee, and saying so is the answer.

"Where do you keep a shopping cart?"
  In the durable store, with Redis in front of it if the read
  volume needs it. A cart in Redis alone disappears on a
  failover, and the user experience of that is a lost cart with
  no error message anywhere.

# An idempotent consumer, with the key derived from the event.

import hashlib
import boto3
from botocore.exceptions import ClientError

table = boto3.resource("dynamodb").Table("processed-events")

def idempotency_key(event: dict) -> str:
    # Derived from the business fact, not from the delivery.
    # order_id + event type is stable across producer retries;
    # the SQS MessageId is not.
    basis = f"{event['order_id']}:{event['event_type']}"
    return hashlib.sha256(basis.encode()).hexdigest()

def handle(event: dict) -> None:
    key = idempotency_key(event)
    try:
        table.put_item(
            Item={"pk": key, "ttl": event["occurred_at"] + 7 * 86400},
            ConditionExpression="attribute_not_exists(pk)",
        )
    except ClientError as exc:
        if exc.response["Error"]["Code"] == "ConditionalCheckFailedException":
            return          # already processed — a successful no-op
        raise

    charge_customer(event)  # the side effect, guarded by the write above

# Two details that matter:
#  1. The conditional write happens BEFORE the side effect, so a
#     crash between them leaves the key claimed. Recovering from
#     that needs a status field, not a bare marker.
#  2. The TTL is longer than any plausible redelivery window, and
#     the record is cheap, so err long.

# One scenario, answered the way it should sound out loud:
# "How would you handle a database migration with no downtime?"

The constraint that shapes everything: during a rolling deploy,
the old and new versions of the code both run against the same
database, at the same time. So every step has to be compatible
with both. That rules out one big migration and produces four
releases instead.

R1  EXPAND
    ALTER TABLE orders ADD COLUMN customer_ref uuid NULL;
    Nullable, no default that rewrites the table, no constraint
    yet. Old code ignores the column; new code has not shipped.

R2  DUAL WRITE
    Deploy code that writes both customer_id and customer_ref,
    and still reads customer_id. Rolling back to R1 is safe:
    R1 code ignores a column it does not know about.

R3  BACKFILL, THEN SWITCH READS
    Backfill in batches — 5,000 rows at a time, with a pause,
    so replication lag and lock time stay bounded. Verify the
    count matches. Then deploy code that reads customer_ref.

R4  CONTRACT
    Once nothing reads customer_id, and only once a rollback to
    R3 is no longer plausible: add NOT NULL, add the foreign
    key, drop the old column.

Why four releases: each one is independently reversible. A
single release that adds the column, backfills and drops the old
one cannot be rolled back at all — the data the old code needs
is gone.