Pods, Services, Deployments

The three workhorses. 90% of interview questions about k8s center here.

Pods

A pod is one or more containers that share:

• A network namespace — same IP, same localhost, ports must not collide.
• Volumes — files visible to every container.
• Lifecycle — they start, restart, and die together.

Almost always one app container per pod. The other-container case is sidecars: log forwarder, Envoy proxy, OAuth2 proxy, init containers.

apiVersion: v1
kind: Pod
metadata: { name: orders, labels: { app: orders } }
spec:
  containers:
  - name: app
    image: my-org/orders:1.0.0
    ports: [{ containerPort: 8000 }]

You almost never write Pod YAML directly. Use a Deployment.

Init containers

Run before app containers; must each exit 0 before the app starts. Good for: waiting on a DB, running schema migrations.

spec:
  initContainers:
  - name: migrate
    image: my-org/orders:1.0.0
    command: ["alembic", "upgrade", "head"]
  containers:
  - name: app
    ...

Migration gotcha: if the deployment rolls out faster than the migration completes, multiple pods race on the migration. Use an argo-workflows Job or a single-pod migration deployment + leader election (advisory lock) rather than init containers when migrations are expensive.

Sidecars

Native sidecar containers (k8s 1.29+) — separate initContainers with restartPolicy: Always. Older clusters use regular containers in the same pod.

spec:
  initContainers:
  - name: log-forwarder
    image: fluent-bit:2.2
    restartPolicy: Always       # native sidecar
  containers:
  - name: app
    ...

Services

A Service is a stable virtual IP + DNS name fronting a set of pods, decoupling clients from the pod IPs that come and go.

apiVersion: v1
kind: Service
metadata: { name: orders }
spec:
  selector: { app: orders }
  ports:
  - port: 80          # the service port
    targetPort: 8000  # the pod port

http://orders from another pod in the same namespace resolves via CoreDNS to the ClusterIP; kube-proxy load-balances to a ready pod via iptables/IPVS/eBPF.

Service types

How traffic actually flows

client → ClusterIP (virtual) → kube-proxy on node (iptables/IPVS/eBPF) → pod IP

The ClusterIP doesn’t exist as a real interface. iptables rules on every node DNAT it to a pod. This is fast but adds a layer of indirection that matters when debugging.

Endpoints

kubectl get endpoints orders shows the actual pod IPs behind the Service. Empty endpoints = no traffic. Common causes:

• Selector doesn’t match any pods.
• Pods are not Ready (failing readiness probe).
• The Service’s targetPort doesn’t match the container’s port.

kubectl get svc orders                              # the service
kubectl get endpointslices -l kubernetes.io/service-name=orders  # endpoints
kubectl describe svc orders                         # selector + ports

Deployments

A Deployment manages a ReplicaSet, which manages Pods. You change the Deployment; it creates a new ReplicaSet; it scales the new one up and the old one down — rolling update.

apiVersion: apps/v1
kind: Deployment
metadata: { name: orders }
spec:
  replicas: 3
  selector:
    matchLabels: { app: orders }
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxSurge: 1          # at most 1 extra pod during the rollout
      maxUnavailable: 0    # never drop below `replicas` ready pods
  template:
    metadata: { labels: { app: orders } }
    spec:
      containers:
      - name: app
        image: my-org/orders:1.0.0
        ports: [{ containerPort: 8000 }]
        readinessProbe:
          httpGet: { path: /healthz/ready, port: 8000 }

Rollout semantics

• maxSurge: 1, maxUnavailable: 0 — safest; takes longest, never below target replicas.
• maxSurge: 25%, maxUnavailable: 25% — k8s default; faster, brief drops below target.
• Each new pod must pass readiness before the next old pod is terminated.

kubectl set image deploy/orders app=my-org/orders:1.1.0   # trigger rollout
kubectl rollout status deploy/orders                       # watch
kubectl rollout history deploy/orders                      # see revisions
kubectl rollout undo deploy/orders                         # rollback
kubectl rollout undo deploy/orders --to-revision=3         # rollback to specific

Why your rollout hangs

• New pods CrashLoopBackOff — image bug.
• Readiness probe fails — pod never reports ready, rollout stuck.
• maxUnavailable: 0 with no spare capacity for maxSurge — scheduler can’t place new pod.
• PodDisruptionBudget blocks termination of old pods.

kubectl describe deploy/orders shows conditions and the immediate cause.

Graceful shutdown

When a pod is terminated:

1. k8s removes it from the Service’s endpoints (eventual).
2. k8s sends SIGTERM to the container.
3. After terminationGracePeriodSeconds (default 30s), SIGKILL.

Your app must:

1. Catch SIGTERM.
2. Stop accepting new requests (uvicorn’s lifespan shutdown).
3. Wait for in-flight to drain.
4. Close DB pool, queue conns.
5. Exit cleanly.

Uvicorn does most of this if you wire the FastAPI lifespan correctly. The classic bug: pod exits before in-flight requests complete → 502s in LB metrics. Add a preStop sleep to avoid the race where the LB hasn’t removed the pod yet:

lifecycle:
  preStop:
    exec: { command: ["sh", "-c", "sleep 5"] }

The 5s gives kube-proxy time to update iptables on every node before the app stops accepting connections.

Putting it together: a real service

---
apiVersion: apps/v1
kind: Deployment
metadata: { name: orders, namespace: prod }
spec:
  replicas: 3
  selector: { matchLabels: { app: orders } }
  strategy: { type: RollingUpdate, rollingUpdate: { maxSurge: 1, maxUnavailable: 0 } }
  template:
    metadata: { labels: { app: orders } }
    spec:
      terminationGracePeriodSeconds: 60
      affinity:
        podAntiAffinity:
          preferredDuringSchedulingIgnoredDuringExecution:
          - weight: 100
            podAffinityTerm:
              labelSelector: { matchLabels: { app: orders } }
              topologyKey: topology.kubernetes.io/zone
      containers:
      - name: app
        image: 1234.dkr.ecr.us-east-1.amazonaws.com/orders:1.0.0
        ports: [{ containerPort: 8000 }]
        resources:
          requests: { cpu: "200m", memory: "256Mi" }
          limits:   { cpu: "1",    memory: "512Mi" }
        readinessProbe:
          httpGet: { path: /healthz/ready, port: 8000 }
          periodSeconds: 5
          failureThreshold: 3
        livenessProbe:
          httpGet: { path: /healthz/live, port: 8000 }
          periodSeconds: 30
          failureThreshold: 3
        lifecycle:
          preStop: { exec: { command: ["sh", "-c", "sleep 5"] } }
---
apiVersion: v1
kind: Service
metadata: { name: orders, namespace: prod }
spec:
  selector: { app: orders }
  ports: [{ port: 80, targetPort: 8000 }]

Anti-affinity spreads replicas across AZs so a zone outage doesn’t kill all replicas. preStop sleep avoids the iptables-race 502s. Liveness has a longer period so transient slowness doesn’t restart the pod.

Interview angle

• “Pod vs Container?” — pod is the k8s scheduling unit, can contain multiple containers sharing network + volumes. Container is the runtime unit. Usually 1 app container per pod.
• “What does a Service actually do?” — gives a set of pods a stable name and virtual IP. kube-proxy on each node DNATs traffic to a ready pod. Decouples clients from pod lifecycle.
• “How does a rolling update work?” — Deployment creates a new ReplicaSet, scales it up while scaling old down, respecting maxSurge and maxUnavailable. Each new pod must pass readiness before the old one is removed.
• “Why is your Service returning 503?” — empty endpoints. Either selector mismatch, pods not Ready, or targetPort doesn’t match the container port. kubectl get endpoints is the first command.
• “What’s the graceful shutdown sequence?” — pod removed from endpoints → SIGTERM → app drains in-flight + closes pools → exit. terminationGracePeriodSeconds sets the SIGKILL deadline. preStop sleep avoids the iptables-propagation race.