Kubernetes
Why are Pods ephemeral?
Short answer
Because a system that can replace a workload cannot also promise it will persist. Ephemerality is not a limitation Kubernetes imposes — it is the price of automatic recovery, and you cannot have one without the other.
The constraint behind it
Kubernetes' central promise is that it keeps your declared state true. You say three replicas; it maintains three replicas, through node failures, evictions, rescheduling and rollouts.
Honouring that promise requires the freedom to destroy and recreate. A node dies, so a pod must be recreated elsewhere. A rollout happens, so old pods must go. The scheduler needs to rebalance, so a pod moves.
Once a system has that freedom, it cannot simultaneously guarantee that any particular pod survives. The two properties are in direct conflict, and Kubernetes resolves it in favour of the workload rather than the instance.
◈ If it were the other way
What that system would look like
Imagine pods were durable — a pod, once created, was expected to persist with its identity and local state intact.
Node failure now becomes a human problem, because nothing can safely recreate the pod elsewhere without knowing what was on its disk. Rollouts require in-place upgrades, which means the old and new versions share a filesystem and a lifecycle. Rescheduling for efficiency becomes impossible, because moving a pod would lose data.
That system exists. It is a fleet of long-lived virtual machines, and it works — with configuration management, snapshots, and a person deciding what to do when one fails. Kubernetes is what you get when you trade that durability for automatic recovery.
◆ What it buys you
- Self-healing that needs no human decision, because there is never a question of whether a replacement is safe to create.
- Rollouts that are just create-and-destroy, which is why a deploy and a recovery use the same machinery.
- Genuine portability: a workload that assumes nothing about its host can be scheduled anywhere with capacity.
- Horizontal scaling as a number rather than a procedure.
◑ What it costs you
- Anything stateful becomes explicit work. Volumes, StatefulSets, PersistentVolumeClaims and their failure modes all exist to reintroduce, carefully, the durability that was traded away.
- Local disk is a cache and nothing more. Every piece of software that assumed otherwise — and a great deal of it did — needs adapting.
- Debugging is harder. The pod that exhibited the problem may not exist by the time you look, which is why centralised logging stops being optional.
Where it bites in practice
Most commonly with file uploads. An application writes to local disk, works perfectly in testing, and starts losing files in production — not on restart, which keeps the same container, but on *replacement*, which is what every deploy does. Nothing errors, because from the new pod's perspective the directory was always empty.
What comes next
Did this get you to an answer?
No text box on purpose — please do not paste production logs anywhere