Container and Kubernetes Concepts

Level: intermediate
Read: 6 min
Concepts: 1

Generic Docker/Kubernetes mechanics — images and containers, pods, deployments, services, ingress, config/secrets, autoscaling — belong to this project's Docker topic and the AWS roadmap's EKS/Kubernetes section, not here; this section does not re-derive them. The one thing left that is genuinely system-design-specific is a judgment call: Kubernetes automates mechanical decisions (how many replicas, how traffic is routed, how a failed container is restarted) once a design already exists, but it makes none of the actual architecture decisions underneath — service boundaries, data ownership, sync-versus-async calls, what happens when a dependency is slow. Fluent kubectl/Helm/YAML use demonstrates that the orchestration layer is being run well; it demonstrates nothing about whether the system underneath it is well designed, and treating the two as equivalent is the mistake this section exists to name.

System Design overview

System Design for Beginners CoursefreeCodeCamp.org

What is true here

Kubernetes automates mechanics — replicas, rollouts, routing, restarts — that assume a design already exists; it does not make service-boundary, data-ownership or failure-handling decisions.
Operating a cluster fluently (kubectl, Helm, YAML) is an operations skill distinct from system design, and answers a different question than a design review is asking.
Generic container/orchestration mechanics are covered by this project's Docker topic and the AWS roadmap's EKS/Kubernetes section — not re-derived in System Design.
A design review question about why a service is split out, or what happens under a dependency's failure, is not answered by describing replica counts or autoscaling policy.

What you will be able to do

Distinguish operating a Kubernetes cluster well from having designed the architecture that cluster runs
Answer a "why is this its own service" or "what happens when X is slow" question with the actual design reasoning, not manifest details
Recognize when a fluent orchestration answer is substituting for architecture reasoning that was never done

Orchestration versus architecture

The one system-design-specific judgment call this section owns, after deferring every generic Docker/Kubernetes mechanic elsewhere.

Orchestration is not architecture

coreintermediate

Kubernetes automates a set of mechanical decisions once they have already been made: how many replicas of a pod to run, how to route traffic between them, how to restart a failed container, how to roll out a new version without downtime. None of that automation makes the underlying design decisions for you — it does not decide whether a service should be split out at all, what data that service owns, which calls are synchronous versus async, where the state lives, or what happens when a downstream dependency is slow or down. Fluency with kubectl, Helm charts and YAML manifests is an operations skill: it tells you the orchestration layer is being run correctly. System design is a different skill: it explains why the system underneath that layer is shaped the way it is, and it is the one Kubernetes cannot supply for you.

Think of it as

Kubernetes is the autopilot on a plane, not the flight plan. Autopilot holds a heading, an altitude and a speed extremely well once someone has decided where the plane is going, when to climb, and when to divert around weather — but ask the autopilot why the flight is routed through a particular corridor and it has no answer, because that was never its job. A pilot who can only fly on autopilot and cannot explain the flight plan is not actually qualified to fly; an engineer who can only operate a Kubernetes cluster and cannot explain the architecture it is running is in the same position.

What we're doing: Contrast an operations-only answer with an architecture answer to the same design-review question.

design-review-question.txttext

Question: "Why is the checkout service
separate from the order service, and what
happens if checkout is slow?"

Operations-only answer:
"They're both Deployments in the same
namespace. Checkout has an HPA scaling on
CPU, three replicas minimum, and an Ingress
routes /checkout to it. If a pod goes
unhealthy the readiness probe pulls it out
of the Service's endpoints."

Architecture answer:
"Checkout is split out because it calls a
third-party payment gateway with unpredictable
latency, and we don't want that latency or
failure blast radius touching order creation.
Checkout writes a pending order and publishes
an event; order service owns the order record
itself. If checkout is slow, orders still get
created -- payment confirmation just arrives
later via the event, and the UI reflects
'processing' rather than blocking."

7: Every fact here is true and can matter operationally — but none of it says why two services exist or what happens to correctness under slowness.
14: This answer names the actual reason for the boundary (blast-radius isolation around unpredictable third-party latency) and the actual failure behavior (eventual consistency via an event), independent of which orchestrator runs it.

Why this works: The two answers describe the exact same running cluster — same Deployments, same Service, same Ingress — but only the second one answers what a design review is actually asking. The Kubernetes facts in the first answer are correct and irrelevant to the question; that gap is the entire point of this concept.

Reaching for manifest details when asked a service-boundary question

Wrong

yaml

# Answering "why is this its own service?"
# by describing the manifest instead of the
# reason the boundary exists
apiVersion: apps/v1
kind: Deployment
metadata:
  name: checkout
spec:
  replicas: 3
  # ...

Better

text

# Answer the boundary question with the
# boundary reason, then cite the manifest
# only as evidence it's actually enforced
"Checkout is isolated because payment
latency is unpredictable and shouldn't
block order creation. The Deployment/HPA
config is just how that isolation gets
run -- 3 replicas, CPU-based autoscaling."

What you see: Asked "why is this its own service" or "what happens when X is slow," the answer describes replica counts, probes, and autoscaling policy in detail but never states a reason the boundary exists or what happens to correctness or user experience under failure — the interviewer or reviewer has to ask a second, more pointed question to get an actual design answer.

Why: YAML and kubectl output describe the current configuration of a decision, not the decision itself. Reciting configuration is a fluent, confident-sounding answer that can pass as expertise in casual conversation, which is exactly why it is a common failure mode — it takes a second, sharper question ("but why does the boundary exist") to expose that no design reasoning was ever behind it.

What Kubernetes automates versus what a design review actually asks

Architecture decisions

service boundaries, data ownership, sync vs. async calls, failure isolation — made by the engineer, before any manifest exists

Orchestration layer (Kubernetes)

replicas, restarts, rollout strategy, service discovery, autoscaling — mechanically enforces whatever shape it is given

Running cluster

kubectl / Helm / YAML — what an operator interacts with day to day

Remember: Kubernetes automates mechanics that already assume a design — replica counts, rollouts, routing, restarts — it does not decide service boundaries, data ownership, or failure behavior. Fluent kubectl/Helm/YAML use answers "is the orchestration layer run well," not "is the system underneath it well designed" — a design review or interview is asking the second question, and reciting manifest details when asked it is the tell that the architecture was never actually reasoned through.