Skip to main content
DevOps Insightsguides & answers

Kubernetes · commonly confused

readinessProbe vs livenessProbe

Short answer

Readiness controls whether traffic is sent to a pod. Liveness controls whether the container is restarted. Getting them the wrong way round is how a slow-starting application ends up in a restart loop.

First published

On this page · 10 sections

In simple terms

Two different questions, asked repeatedly. Readiness asks “should I send requests here right now?” and a no takes the pod out of the Service's endpoint list. The container keeps running, but a rollout also counts the pod as unavailable until it is ready again. Liveness asks “is this container beyond saving?” and the consequence of a no is a restart.

So readiness failing is routine and safe — it happens on every start-up, and during any period when the application is temporarily unable to serve. Liveness failing is not routine. It kills the container.

The rule that follows: if a condition might resolve on its own, that is a readiness concern. Only use liveness for states a restart genuinely fixes, like a deadlock the process cannot recover from.

Words used on this page

probe
A check the kubelet runs against a container on a schedule. It can run a command (exec), make an HTTP GET, open a TCP connection or call a gRPC health check.
kubelet
The agent on each node that starts a Pod's containers and runs their probes. Probes are not run by the control plane.
Ready condition
A true or false flag in the Pod's status. A failing readiness probe sets it to false. Services and Deployment rollouts both read it.
EndpointSlice
The list of Pod addresses a Service sends traffic to. A Pod that is not ready stops getting traffic through it.
startup probe
A third probe that runs only while the container starts. Liveness and readiness probes wait until it has succeeded once.
failureThreshold
How many failures in a row count as a failed check. Defaults to 3.
periodSeconds
How often the probe runs. Defaults to 10 seconds.
initialDelaySeconds
How long to wait after the container starts before the first probe. Defaults to 0.
restartPolicy
The Pod's rule for restarting containers: Always, OnFailure or Never. A failed liveness probe kills the container; this policy decides whether it comes back.
CrashLoopBackOff
The status kubectl shows while the kubelet waits before restarting a container that keeps failing. By default the wait starts at 10 seconds and doubles, up to 5 minutes.

What actually happens

Both are run by the kubelet on the node, not by the control plane. A readiness failure causes the pod's Ready condition to go false, and the EndpointSlice controller removes its address from the EndpointSlices of every Service that selects it, so traffic through them stops. The kubelet takes no other action: the container keeps running and keeps being probed. A rollout does wait for the pod, because a pod that is not ready does not count as available.

A liveness failure past failureThreshold causes the kubelet to kill the container. The restart counter increments, and the backoff is exponential, which is where CrashLoopBackOff comes from.

startupProbe exists specifically to separate these concerns. While a startup probe is running, liveness and readiness are both suspended, so a slow start is not mistaken for a hang, as long as the startup probe's failureThreshold times periodSeconds covers the slowest start. A startup probe that runs out kills the container too. If you find yourself setting a large initialDelaySeconds on liveness, a startup probe is what you actually wanted.

Probes that will not fight each other

# Give start-up its own probe so a slow boot is not read as a hang.
startupProbe:
  httpGet: { path: /healthz, port: 8080 }
  failureThreshold: 30
  periodSeconds: 2        # allows up to 60s to start

readinessProbe:
  httpGet: { path: /ready, port: 8080 }
  periodSeconds: 5        # cheap; may include dependency checks

livenessProbe:
  httpGet: { path: /healthz, port: 8080 }
  periodSeconds: 20
  failureThreshold: 3     # ~60s of failure before a restart

Note the two different paths. /ready may check that the database is reachable; /healthz must not. A liveness probe that fails when a dependency is down will restart a perfectly healthy container, repeatedly, for a problem it cannot fix.

Try it yourself

Before you start

  • A Kubernetes cluster you can safely break things in, and kubectl configured for it: kubectl version should print a server version as well as the client one. The Kubernetes task page suggests minikube or an online playground if you have no cluster. Use a test namespace, never production.
  • Permission to create Pods and read their events in that namespace. kubectl auth can-i create pods -n <namespace> should print yes.
  • Nodes that can pull from registry.k8s.io, because the example uses the registry.k8s.io/busybox:1.27.2 image.
  • The example below is quoted from the Kubernetes v1.37 documentation, not run here. Field names and defaults on this page are from that version; check yours with kubectl version.
  • For your own application: how long it takes to start at worst, and two endpoints. One should be cheap and report only the process's own health, for liveness. The other may also check the services it depends on, for readiness.

Watching a liveness probe restart a container

Quoted from the Kubernetes task page Configure Liveness, Readiness and Startup Probes; not reproduced here, because no cluster was available. The Pod was created with kubectl apply -f https://k8s.io/examples/pods/probe/exec-liveness.yaml. Its busybox container creates /tmp/healthy, deletes it 30 seconds later, and has a liveness probe that runs cat /tmp/healthy every 5 seconds after a 5-second delay. The page shows only the Events section at the bottom of kubectl describe, taken about 35 seconds after the Pod started, then kubectl get pod about 30 seconds later.

Command

kubectl describe pod liveness-exec
kubectl get pod liveness-exec

Output · as shown in Kubernetes: Configure Liveness, Readiness and Startup Probes

$ kubectl describe pod liveness-exec
Type     Reason     Age                From               Message
----     ------     ----               ----               -------
Normal   Scheduled  57s                default-scheduler  Successfully assigned default/liveness-exec to node01
Normal   Pulling    55s                kubelet, node01    Pulling image "registry.k8s.io/busybox:1.27.2"
Normal   Pulled     53s                kubelet, node01    Successfully pulled image "registry.k8s.io/busybox:1.27.2"
Normal   Created    53s                kubelet, node01    Created container liveness
Normal   Started    53s                kubelet, node01    Started container liveness
Warning  Unhealthy  10s (x3 over 20s)  kubelet, node01    Liveness probe failed: cat: can't open '/tmp/healthy': No such file or directory
Normal   Killing    10s                kubelet, node01    Container liveness failed liveness probe, will be restarted

$ kubectl get pod liveness-exec
NAME            READY     STATUS    RESTARTS   AGE
liveness-exec   1/1       Running   1          1m

How to read it

  • The Unhealthy warning was recorded three times (x3). The manifest sets no failureThreshold, so the default of 3 failures in a row applies, and only then does the kubelet act.
  • Everything after Liveness probe failed: is the output of the probe itself, here the error from cat. Read it first: it tells you what the check saw, not what the application logged.
  • Killing with failed liveness probe, will be restarted is the step only liveness and startup probes take. A failing readiness probe records the same kind of Unhealthy warning, starting Readiness probe failed:, but no Killing event: the container keeps running and the Pod's Ready condition goes false.
  • RESTARTS 1 with Running: the container was restarted in place, inside the same Pod. Its command creates the file again on each start and deletes it 30 seconds later, so it will keep failing and the count will keep rising.
  • READY 1/1: this Pod has no readiness probe, and the kubelet treats a missing probe as a success. So it counted as ready while its liveness check was failing. A liveness probe alone does not take a Pod out of a Service; readiness does that.

The mistake this causes

Pointing the liveness probe at an endpoint that checks dependencies.

Why people do it It seems more thorough. If the database is down, the application cannot work, so surely it is unhealthy? But liveness answers “would a restart help?”, and restarting your API does not fix someone else's database.

What you see A dependency stays down for longer than the liveness probe tolerates, and every replica restarts within moments of the others — turning a partial degradation into a total one, and adding a thundering herd of reconnections at exactly the moment the dependency is struggling. Restart counts climb across every pod at once, which is the signature of a shared cause rather than a pod-level fault.

If it goes wrong

Liveness probe failed: HTTP probe failed with statuscode: 500, then Container <name> failed liveness probe, will be restarted
The liveness endpoint answered with an error, so the kubelet is restarting the container. Run kubectl logs <pod> --previous to read the output of the container that was killed, and look at what the endpoint checks. If it fails because a database or another service is down, move that check to the readiness probe: a restart cannot fix it.
Restarts climb during start-up, and the events show Liveness probe failed before the application has finished starting
The liveness probe starts before the application can answer. Add a startupProbe on the same endpoint, with failureThreshold times periodSeconds longer than your slowest start, and leave the liveness settings alone. If the events then say Container <name> failed startup probe, will be restarted, that budget is still too short.
The Pod shows 0/1 under READY and Running under STATUS, and the events show Readiness probe failed:
The container is running but gets no Service traffic, which is readiness doing its job. Check the probe's path and port against the application. The kubelet probes the Pod's IP address, so an application listening only on 127.0.0.1 fails every HTTP and TCP probe. If every replica is unready at once, the Service has no ready endpoints; kubectl get endpointslices -l kubernetes.io/service-name=<service> shows what it has.
kubectl apply is rejected with an error on livenessProbe.successThreshold that ends must be 1
Liveness and startup probes only accept a successThreshold of 1. Remove the field or set it to 1. Only a readiness probe can require several successes in a row.
kubectl apply is rejected with an error on readinessProbe.terminationGracePeriodSeconds that ends must not be set for readinessProbes
A per-probe grace period only applies to probes that stop the container. Move it to the liveness or startup probe, or set terminationGracePeriodSeconds on the Pod instead.

How it shows up in production

Another version has nothing to do with dependencies. An application takes 40 seconds to start — JIT warm-up, migrations, a cache load. The liveness probe has initialDelaySeconds: 10. So at 10 seconds the probe fails, at 30 seconds the kubelet kills the container, and it starts again from zero.

It never gets far enough to become healthy, and the restart loop is caused entirely by the probe. The logs show a clean start-up sequence cut off mid-way with no error, which is a genuinely confusing thing to look at: nothing is wrong with the application, and nothing in its output says so.

How to tell which one you are hitting

Is the restart count climbing with no error in the logs?
Suspect the liveness probe. Check kubectl describe pod for Liveness probe failed events, and compare the probe's timing against how long start-up actually takes.
Did every pod restart at roughly the same moment?
A pod-level fault does not synchronise. Simultaneous restarts across a fleet point at a shared cause, such as a dependency checked by liveness, or load heavy enough that every pod's liveness probe fails at once.
Does the pod work when you remove the liveness probe?
A decisive test in a non-production namespace. If it becomes healthy, the probe was the fault and the fix is timing or a startupProbe, not the application.

Security considerations

  • A liveness probe that checks a shared dependency turns that dependency's outage, or a load spike, into restarts of every replica. The Kubernetes docs warn that badly designed liveness probes can cause cascading failures: restarts under high load, failed requests, and more work for the Pods that are left. Keep liveness to the process's own state.
  • Probe endpoints are served on the Pod's IP address, so anything that can reach the Pod can call them, not only the kubelet. Keep /healthz and /ready free of versions, hostnames and error details, and never let them change state. Where your network plugin enforces NetworkPolicy, a policy can keep other Pods out without breaking probes, because traffic from the Pod's own node is always allowed.
  • An HTTPS probe does not check the certificate: the kubelet skips verification. A passing probe says nothing about whether clients will accept the certificate, so an expired one does not fail the probe and the Pod stays in the Service.
  • An exec probe runs its command inside your container on every period, as the container's user, and forks new processes each time. The command is part of the Pod spec, so anyone who can read the Pod can read it: never put a password or token in it.

In an interview

How this gets asked in an interview

Usually phrased as “What is the difference between a readiness and a liveness probe?”

What a strong answer contains Give the consequence of each failing rather than the definition: readiness removes the pod from Service endpoints, liveness restarts the container. Then state the rule — if the condition might resolve on its own, it is readiness — and mention startupProbe as the correct tool for slow starts.

The follow-up “What happens if the liveness probe checks the database?” is the follow-up worth being ready for, because the answer is a fleet-wide restart during a dependency outage.

Sources

The behaviour this page describes, as documented by the projects themselves.

Terms used here (4)

Short definitions for the vocabulary on this page, in case any of it is new. Each links to the full entry.

Canary Release
A deployment technique where a new version is rolled out to a small subset of users before being promoted to the entire fleet. This limits the blast radius of bugs and allows teams to monitor real-world performance before full rollout.
Health Check
An endpoint or mechanism that reports whether a service is running correctly. Health checks are used by load balancers, orchestrators, and monitoring systems to detect failures and automatically route traffic away from unhealthy instances.
Liveness Probe
A Kubernetes health check that determines when a container needs restarting — for example, when its process is still running but deadlocked and unable to make progress. If the probe fails failureThreshold times in a row (3 by default), the kubelet kills the container and restarts it according to the pod's restart policy. A carelessly designed liveness probe can cause restarts of its own.
Readiness Probe
A Kubernetes health check that determines whether a container is ready to accept traffic. Unlike liveness probes, a failing readiness probe doesn't restart the container — it simply removes the Pod from the Service's endpoint list until it recovers.

All 59 terms in the glossary

Did this get you to an answer?

No text box on purpose — please do not paste production logs anywhere

Something wrong or out of date? Report it