Understand
What happens if…
Knowing what a component is gets you surprisingly far and then stops. The questions that actually come up during an incident are sequential: what happened first, what is happening now, what is about to happen, and which part of this system is responsible for it.
So these are written as timelines rather than explanations, and every step names its actor. That last part is the point — most confusion about distributed systems comes from misattribution. People blame Kubernetes for something the kernel did, or their application for something the network did. Reading down the actor column teaches the division of responsibility, which outlasts any individual detail.
Timings are documented defaults and are labelled as such. A cluster that changed them will behave differently, and knowing which numbers are configurable is part of knowing the system.
- Kubernetes · 7 stepsWhat happens if a Pod is deleted while it is serving traffic?Two things start at once and neither waits for the other: the pod begins shutting down, and its address begins being removed from Services. Requests that arrive in the gap between those get sent to a container that is already stopping.
- Kubernetes · 7 stepsWhat happens if a container exceeds its memory limit?Once the kernel cannot reclaim enough memory to stay under the limit, it kills processes in the container, and the container ends when its main process dies. There is no warning, no signal your application can catch, and no chance to shut down cleanly:
limits.memoryis a hard limit, enforced by the OOM killer rather than by slowing the process down. - Kubernetes · 8 stepsWhat happens if a Kubernetes node suddenly disappears?More slowly than people expect. The control plane cannot distinguish a dead node from an unreachable one, so it waits: about 50 seconds (40 before Kubernetes 1.32) before the node's pods are marked not ready and taken out of Service endpoints, and about five more minutes before they are evicted. A Deployment then replaces them elsewhere; a StatefulSet's pods wait until the node returns or someone acts. Your callers feel the first part before Kubernetes notices anything.
- Kubernetes · 7 stepsWhat happens if DNS stops resolving inside the cluster?Almost everything breaks, in a way that looks like several unrelated failures at once — because service discovery is a dependency of nearly every call, and DNS caching means it breaks unevenly.
- Kubernetes · 5 stepsWhat happens if a deployment reports success but users still see errors?It means every check that ran was satisfied — and none of them checked the thing that is broken. A successful rollout proves that pods started and passed their probes, which is a much weaker claim than "the application works".