Kubernetes · what happens if

What happens if a Pod is deleted while it is serving traffic?

Short answer

Two things start at once and neither waits for the other: the pod begins shutting down, and its address begins being removed from Services. Requests that arrive in the gap between those get sent to a container that is already stopping.

First published 4 Sept 2026

On this page · 8 sections

Starting state

A Deployment with three replicas behind a Service. One pod is deleted — by a rollout, a node drain, an API-initiated eviction or kubectl delete. For these the mechanism does not matter; what follows is the same. A node-pressure eviction by the kubelet is different: it ignores terminationGracePeriodSeconds, and for a hard threshold it kills the pod with no grace period.

Words used on this page

deletion timestamp: The time the API server records on a Pod when you delete it gracefully. The Pod stays in the API, shown as Terminating, until shutdown finishes.
grace period: terminationGracePeriodSeconds, 30 by default: how long the preStop hook and the container's own shutdown get, together, before the remaining processes are killed.
SIGTERM: The signal that asks a process to shut down. A process can catch it, finish its work and exit.
SIGKILL: The signal that ends a process immediately. It cannot be caught, blocked or ignored, so nothing gets cleaned up.
preStop hook: A command, HTTP request or sleep that Kubernetes runs in or for a container before it sends the stop signal. It must finish before the signal is sent.
EndpointSlice: An API object that lists the Pod addresses behind a Service, each with conditions saying whether it should receive traffic.
ready, serving, terminating: An endpoint's conditions. A terminating endpoint always has ready false; serving says whether the Pod itself would still answer.
kube-proxy: The component on each node that turns Services and EndpointSlices into the node's packet-forwarding rules.
PodDisruptionBudget: A limit on how many Pods of an application may be evicted at once through the Eviction API, for example by kubectl drain.
node-pressure eviction: The kubelet itself ending Pods because the node is short of memory, disk or other resources. It does not follow the Pod's grace period.

The sequence

Colour marks which component acts. Steps sharing a timestamp happen concurrently — nothing here waits for anything else unless it says so.

control plane
kubelet
network
your app

t+03 things at once
- control plane
  The pod's deletionTimestamp is set. It is not removed from the API yet — it is marked as going away.
  What you would see kubectl get pod shows status Terminating.
- control plane
  In parallel, the EndpointSlice controller marks the pod's endpoint as no longer ready. Since Kubernetes 1.26 the address stays in the slice, flagged terminating, until the pod object is gone, but kube-proxy stops sending new connections to it as long as another endpoint is ready. If none is, it falls back to endpoints that are terminating but still serving.
  What you would see kubectl get endpointslice -o yaml shows the address with ready: false and terminating: true. A plain kubectl get endpointslices still lists it.
- kubelet
  Also in parallel, the kubelet runs any preStop hook and then has the container runtime send SIGTERM to PID 1 in each container. SIGTERM is the default: many runtimes send the image's STOPSIGNAL instead when it sets one.
shortly after t+0network
Every node's kube-proxy (or CNI equivalent) has to see the endpoint change and update its own rules. There is no fixed time for this: in iptables mode, for example, kube-proxy may hold a change for up to its minSyncPeriod, 1 second by default, before applying it. This is where the gap lives: the pod is already receiving SIGTERM while some nodes still consider it a valid destination.
What you would see A small burst of connection errors or resets at exactly the moment of a rollout, which can look like a load balancer fault.
t+0 to grace periodyour app
Your application decides what SIGTERM means. If it stops accepting new connections and finishes in-flight requests, this is graceful. If it exits immediately, in-flight requests are dropped. If it ignores the signal, nothing happens yet.
t+30s (default)kubelet
The grace period expires and any container still running receives SIGKILL. There is no negotiation and no cleanup.
What you would see Exit code 137, and a pod that took the full grace period to disappear.
after terminationcontrol plane
The pod object is removed. If it was managed by a ReplicaSet, the controller has already created a replacement to restore the desired replica count.

What you actually control

Separated from what happens regardless, because conflating the two is how people end up tuning the wrong thing.

terminationGracePeriodSeconds: How long the preStop hook and the response to SIGTERM get, together, before SIGKILL. If the hook is still running when it expires, the kubelet allows a one-off 2 seconds more. Raise it if your application legitimately needs longer to drain; lowering it makes deploys faster and drops more requests.
A preStop hook with a short sleep: A mitigation for the endpoint-propagation gap. Sleeping a couple of seconds before SIGTERM gives the network time to stop routing to the pod, so the requests that would have landed in the gap go elsewhere instead. Kubernetes has a built-in sleep action for this hook, on by default since 1.30 and stable since 1.34, so the image does not need a sleep binary. The sleep counts against terminationGracePeriodSeconds.
Handling SIGTERM in your application: The part nothing else can do for you. Stop accepting new work, finish what you have, close connections, exit. Without this, none of the above helps.
A PodDisruptionBudget: Does not change any of this sequence — it limits how many pods may be in it at once during evictions through the Eviction API, such as a node drain. Deleting a pod directly and a Deployment rollout are not limited by it.

Try it yourself

Before you start

A Kubernetes cluster that is not production, and kubectl configured for it. The EndpointSlice terminating and serving conditions are stable from Kubernetes 1.26, and the built-in sleep action for a preStop hook is on by default from 1.30.
Permission in a namespace of your own to create a Deployment and a Service, delete Pods, and read EndpointSlices.
The tutorial's two manifests, a one-replica nginx Deployment and the Service nginx-service. Its Deployment writes sleep: 180 directly under preStop; the Pod API reference defines the sleep action as an object with a required seconds field, so write it as sleep: {seconds: 180}.
Two terminals, so you can run kubectl get pods and read the EndpointSlice while the deleted Pod is still terminating. With the tutorial's 120-second grace period you have two minutes.

A deleted Pod that is still serving

Quoted from the Kubernetes tutorial Explore Termination Behavior for Pods And Their Endpoints; not reproduced here, because no cluster was available. The tutorial runs a one-replica nginx Deployment behind the Service nginx-service, with terminationGracePeriodSeconds: 120 and a preStop sleep, so the Pod keeps serving while it terminates. Both commands ran shortly after kubectl delete pod nginx-deployment-7768647bf9-b4b9s. The JSON is an excerpt: the deleted Pod's entry in the EndpointSlice, cut after the Pod's name. The tutorial shows output as "similar to this".

Command

kubectl get pods
kubectl get endpointslice -o json nginx-service-6tjbr

Output · as shown in Kubernetes — Explore Termination Behavior for Pods And Their Endpoints

$ kubectl get pods
NAME                                READY   STATUS        RESTARTS      AGE
nginx-deployment-7768647bf9-b4b9s   1/1     Terminating   0             4m1s
nginx-deployment-7768647bf9-rkxlw   1/1     Running       0             8s

$ kubectl get endpointslice -o json nginx-service-6tjbr
            "addresses": [
                "10.12.1.201"
            ],
            "conditions": {
                "ready": false,
                "serving": true,
                "terminating": true
            },
            "nodeName": "gke-main-default-pool-dca1511c-d17b",
            "targetRef": {
                "kind": "Pod",
                "name": "nginx-deployment-7768647bf9-b4b9s",

How to read it

The deleted Pod shows Terminating with READY 1/1: it has a deletion timestamp but is still running, held open by its preStop sleep. Terminating is what kubectl prints for such a Pod; it is not one of the Pod phases.
The replacement, nginx-deployment-7768647bf9-rkxlw, is already Running and 8 seconds old while the old Pod is still there. The ReplicaSet stopped counting the old Pod as soon as its deletion began, so for a while there are more Pods than replicas.
The old address, 10.12.1.201, is still in the EndpointSlice with ready: false, serving: true and terminating: true. ready is always false for a terminating endpoint, so kube-proxy stops choosing it for new connections while a ready endpoint exists; serving: true tells a load balancer that drains connections that the Pod would still answer.
The address leaves the EndpointSlice only when the Pod object is deleted, at the end of the grace period. The ready: false change, though, reaches each node's packet rules only when that node's kube-proxy applies it, and in iptables mode a change can wait up to minSyncPeriod, 1 second by default. That wait is the gap this page is about.

Where this goes wrong

The application ignores SIGTERM
Every pod takes the full grace period to die and is then killed. Deploys are slow, and any request still in flight when SIGKILL arrives is dropped. One cause is a shell-form ENTRYPOINT making /bin/sh PID 1; another is an application running as PID 1 without a SIGTERM handler, because the kernel delivers a signal to a namespace's first process only if it has a handler for it.
There is no preStop delay and traffic is high
A reliable trickle of errors on every deploy, proportional to request rate. Hard to attribute because each individual error looks like a transient network fault.
The grace period is shorter than the work in flight
Long-running requests are cut off mid-response. For a background worker, a job can be lost silently unless it is idempotent and re-queued.

If it goes wrong

Every Pod stays Terminating for the whole grace period on each rollout: The process that receives the stop signal is not acting on it. With a shell-form ENTRYPOINT, /bin/sh -c is PID 1 and does not pass signals on: use the exec form, or exec the program at the end of a start script. A program that is PID 1 must also install a SIGTERM handler, because the kernel only delivers a signal to a PID namespace's first process if it has a handler for it.
Connection errors or resets at each rollout, although the application handles SIGTERM: That is the endpoint-propagation gap. Add a short preStop sleep (sleep: {seconds: N}) and keep serving while it runs, then shut down on SIGTERM. Raise terminationGracePeriodSeconds so it covers the sleep plus your longest drain, since the two share it.
kubectl drain keeps retrying, and the Eviction API answers 429 Too Many Requests: A PodDisruptionBudget does not allow another eviction yet. Wait for the replacement Pods to become Ready. If they never do, the API keeps answering 429 until you intervene, so fix the replacement rather than deleting the budget.
Pods evicted because a node ran short of memory or disk got no graceful shutdown: Node-pressure eviction ignores terminationGracePeriodSeconds and PodDisruptionBudgets. With a hard threshold the kubelet uses a 0-second grace period; with a soft one it uses eviction-max-pod-grace-period if the administrator set it. Set requests that match real usage so your Pods are not the ones over their requests when the node runs short.

The part worth remembering

The generalisable point is that shutdown is not a single event with a single owner. Four different components act, they act concurrently, and none of them coordinates with the others. Kubernetes does not stop traffic and then signal the pod; it does both at once and lets the timing work itself out.

Once you see that, a whole class of "transient errors during deploys" stops being mysterious. It is not a bug in the load balancer and not flakiness — it is a race that exists by design, and the mitigations are all about giving the slower participant a head start.

Security considerations

Avoid kubectl delete --grace-period=0 --force to speed things up. The API server removes the Pod without waiting for the kubelet to confirm the containers stopped, so a replacement can start while the old process still runs on an unreachable node. For a StatefulSet, the documentation warns this can break its at-most-one guarantee and lead to data loss.
SIGKILL allows no clean-up. Temporary files holding secrets, locks and half-written uploads stay wherever the container wrote them, and a volume outlives the container. Handle SIGTERM, and give the grace period room to finish that clean-up.
An exec preStop hook runs inside the container, with its privileges and against its resource limits, while sleep and httpGet hooks are run by the kubelet. Prefer the built-in sleep action over shipping a sleep binary or a shell in an image that otherwise would not need one.
Node-pressure eviction skips grace periods and disruption budgets, so a workload that exhausts a node's memory can cause abrupt kills of other teams' Pods. Set memory requests and limits on every workload, as the page on a container exceeding its memory limit explains.

Sources

The behaviour this page describes, as documented by the projects themselves.

Did this get you to an answer?

No text box on purpose — please do not paste production logs anywhere

Something wrong or out of date? Report it