Skip to main content
DevOps Insightsguides & answers

Kubernetes · troubleshooting

Service has no endpoints

The Service exists and its name resolves, but nothing is behind it, so connections to it have nowhere to go. Almost always the selector, the readiness of the pods, or the port — and which one it is takes two commands to establish.

First published

New to this? Read what is actually happening first, then come back to the steps.

On this page · 10 sections

Before you start

  • kubectl access to the Service's namespace with read permission on services, pods and endpointslices (API group discovery.k8s.io). kubectl auth can-i list endpointslices.discovery.k8s.io -n <namespace> tells you whether you have the last one.
  • The namespace the workload really runs in. Pass -n <namespace> on every command, because a Service only selects pods in its own namespace.
  • A way to send a request from inside the cluster: permission to kubectl exec into a running pod whose image has wget or curl, or to start a short-lived test pod as the Debug Services page does with kubectl run -it --rm.
  • A cluster on Kubernetes v1.21 or later, where EndpointSlices are stable. From v1.33 the older Endpoints API is deprecated, and kubectl get endpoints prints a warning.
  • Write access to the Service, or to the Deployment that owns the pods, and agreement from their owner. Any change to a Deployment's pod template starts a rollout that replaces its pods.

Run this first

step 1 of 4
kubectl get endpointslices -l kubernetes.io/service-name=<service>

Look for ENDPOINTS showing <unset>, or every address with ready: false in -o yaml

Confirms the symptom and tells you immediately whether this is the problem you have. EndpointSlices replace the deprecated Endpoints API, and list pods that are not Ready too, so read ready for each address. An empty list here is the whole diagnosis of where the fault is not.

Work out which cause you have

A few questions to narrow the list. Every answer ends in a command that confirms or rules the cause out — this cannot see your cluster, so nothing here is a certainty until you have checked.

Narrow it down

0 answered · nothing is sent anywhere

Does kubectl get pods -l <your selector> return any pods at all?

Use the exact selector from the Service, not the one you meant to write.

Cause space

5 of 5 still possible

  • The selector does not match the pod labelsCommon
  • The pods match but none are ReadyCommon
  • targetPort does not match the container portCommon
  • The pods are in a different namespaceOccasional
  • There are no pods at allOccasional

Nothing ruled out yet. Answer the question above and the branches your answer eliminates will strike through here.

Check it with a tool

Or diagnose it manually

In this order. The first command usually contains the whole answer.

  1. Step 1

    kubectl get endpointslices -l kubernetes.io/service-name=<service>

    Confirms the symptom and tells you immediately whether this is the problem you have. EndpointSlices replace the deprecated Endpoints API, and list pods that are not Ready too, so read ready for each address. An empty list here is the whole diagnosis of where the fault is not.

    Look for ENDPOINTS showing <unset>, or every address with ready: false in -o yaml

  2. Step 2

    kubectl get svc <service> -o jsonpath='{.spec.selector}'; echo; kubectl get pods --show-labels

    Puts the selector next to the labels that exist, which resolves the most common cause by reading rather than guessing.

    Look for A key or value in the selector that no pod carries

  3. Step 3

    kubectl get pods -l app=<label> -o wide

    Separates 'no pods match' from 'pods match but are not Ready'. These look identical at the Service and need entirely different fixes.

    Look for READY 0/1 with STATUS Running, which points at the readiness probe

  4. Step 4

    kubectl describe svc <service>

    Shows the resolved ports and any events the Service itself recorded.

    Look for TargetPort, and an Endpoints line confirming what the control plane computed

Every cause, and how to fix it

Ordered by how often each one turns out to be the answer.

The selector does not match the pod labels

Common

The Service's selector and the pod template's labels differ — a typo, a renamed app, or a selector copied from another manifest. A subtler version: the selector was written against the labels on the Deployment object rather than the labels inside spec.template.metadata.labels, which are what the pods actually carry.

Confirm

kubectl get pods -l "$(kubectl get svc <service> -o jsonpath='{.spec.selector}' | tr -d '{}"' | tr ':' '=')" --show-labels

No pods returned. Then compare against kubectl get pods --show-labels to see what labels the pods really carry

Fix

  • Make the Service selector match spec.template.metadata.labels in the Deployment exactly, key and value.
  • Change the Service rather than relabelling running pods. A label added by hand is lost when the pod is replaced, and changing a label the ReplicaSet selects on orphans the pod and makes the ReplicaSet start a replacement.
  • Keep the selector minimal. Matching on one stable label is more robust than matching on five that include a version.
kubectl get svc <service> -o jsonpath='{.spec.selector}'; echo; kubectl get deploy <deployment> -o jsonpath='{.spec.template.metadata.labels}'

Prints the two things that must agree, next to each other.

The pods match but none are Ready

Common

A matching pod is listed in the EndpointSlice straight away but marked ready only once it passes its readiness probe, and only ready addresses get traffic. If the probe checks the wrong path or port, the pods run indefinitely and never receive traffic. A slow start alone does not cause this, because the readiness probe keeps running for the container's whole life; a liveness probe that kills the container before it is up does, and shows as a climbing RESTARTS count. The Service is behaving exactly as designed; the probe is the thing that is wrong.

Confirm

kubectl get pods -l app=<label> -o wide

READY showing 0/1 while STATUS is Running. Then kubectl describe pod <pod> for the probe failure message

Fix

  • Check the readiness probe's path, port and scheme against what the application actually serves.
  • For a slow starter, add a startup probe so the liveness probe cannot restart the container before it has started. Liveness and readiness probes wait until the startup probe succeeds, and the docs accept either initialDelaySeconds or a startup probe for delaying a readiness probe.
  • Confirm the probe from inside the pod before changing the manifest, so you are testing the application rather than guessing.

targetPort does not match the container port

Common

The Service's port is what clients call; targetPort is where it forwards on the pod. When targetPort names a port the container does not listen on, endpoints can still appear and every connection fails. When targetPort uses a name, that name must be defined, with the same protocol, in the container's ports list — an undefined name resolves to nothing.

Confirm

kubectl get svc <service> -o jsonpath='{.spec.ports[*]}{"\n"}'; kubectl get pod <pod> -o jsonpath='{.spec.containers[*].ports[*]}{"\n"}'

targetPort against containerPort, and whether a named targetPort exists in the container's ports

Fix

  • Set targetPort to the port the process actually binds, which is not always the one documented.
  • Prefer named ports and define the name on the container, so the two sides cannot drift numerically.
  • Verify what the process listens on from inside the pod rather than from the manifest.

The pods are in a different namespace

Occasional

A Service selects only within its own namespace. Applying a Service to default while the workload runs in production produces a valid Service with an empty endpoint list and no error anywhere. Common after copying a manifest between environments or forgetting -n on an apply.

Confirm

kubectl get svc <service> -o jsonpath='{.metadata.namespace}{"\n"}'; kubectl get pods -A -l app=<label> -o custom-columns=NS:.metadata.namespace,NAME:.metadata.name

The Service's namespace differing from where the matching pods actually run

Fix

  • Apply the Service into the namespace the workload runs in.
  • For genuine cross-namespace access, address the Service by its fully qualified name from the other namespace rather than trying to select across the boundary.

There are no pods at all

Occasional

The Service is correct and the workload is not running — scaled to zero, every pod stuck Pending, or a Deployment that failed to roll out. The endpoint list is empty for the simplest possible reason, and looking at the Service wastes time that belongs on the workload.

Confirm

kubectl get deploy -n <namespace>; kubectl get rs,pods -n <namespace> -l app=<label>

A Deployment showing 0 available replicas, or no pods listed at all

Fix

  • Establish why the workload is not running before touching the Service — see the Pending and CrashLoopBackOff pages.
  • Check the replica count has not been scaled to zero by an autoscaler or a previous mitigation.

Understanding it properly

Skip this if you are mid-incident — the working part of the page is above. Worth reading afterwards, because understanding the mechanism is what stops the next one.

In plain terms

A Service gives a group of pods one stable name and address. It finds those pods by their labels, keeps a list of their addresses called endpoints, and sends traffic only to the pods that pass their health check. When no healthy pod is on that list, the name still exists but nothing answers, and your job is to find which link broke: the labels, the health check, the port, or the pods themselves.

What a working Service's endpoints look like

Output quoted from the Kubernetes Debug Services task page. There, a Deployment named hostnames runs three replicas that serve HTTP on port 9376, and a Service maps port 80 to them. No cluster was run for this page, so compare your own output with this one.

Command

kubectl get endpointslices -l kubernetes.io/service-name=hostnames

Output · as shown in Kubernetes — Debug Services

NAME              ADDRESSTYPE   PORTS   ENDPOINTS
hostnames-ytpni   IPv4          9376    10.244.0.5,10.244.0.6,10.244.0.7

How to read it

  • You ask for slices by the kubernetes.io/service-name label, not by name, because a Service can have several slices and their names are generated.
  • PORTS shows 9376, the pod port from targetPort, not the Service's port 80. If the container does not listen on this number, addresses still appear here and every connection still fails.
  • ENDPOINTS lists every matching pod that has an IP, ready or not. Add -o yaml and read conditions.ready for each address; kubectl describe svc shows only the ready ones on its Endpoints line.
  • When the selector matches no pod at all, the slice still exists and current Kubernetes prints <unset> under both PORTS and ENDPOINTS. The Debug Services page says to look for <none>, which is how the older Endpoints listing prints an empty list.

Words used on this page

Service
A stable name and virtual IP address for a changing set of pods. Clients call the Service, and Kubernetes forwards each connection to one of the ready pods behind it.
Selector
The labels a Service looks for, in spec.selector. A pod is selected only if it carries every one of those labels with the same value.
Label
A key and value attached to an object, such as app: hostnames. Pods get theirs from the Deployment's spec.template.metadata.labels, not from the Deployment's own metadata.labels.
EndpointSlice
The object where the control plane lists the addresses of the pods a Service selects, each with a ready condition. You find a Service's slices by the label kubernetes.io/service-name=<service>.
Ready
A pod condition that turns true when its containers pass their readiness probes. Only ready addresses receive Service traffic; the others stay listed with ready: false.
port and targetPort
port is the number clients call on the Service. targetPort is where the Service forwards on the pod, given as a number or as the name of a port defined on the container.
Namespace
A named partition of the cluster. A Service only ever selects pods in its own namespace.
What is actually happening

A Service does not forward traffic by itself. It selects pods by label, and the control plane maintains the resulting list of addresses as EndpointSlices. When that list is empty, the Service still exists and DNS still resolves a normal Service's name to its cluster IP, but connections to it fail: kube-proxy in iptables mode installs a rule that rejects them. A headless Service has no cluster IP; its name resolves to the selected pods' addresses, so with no pods there is nothing to return.

That is why the symptom appears everywhere except the Service. An Ingress in front reports 503. A client inside the cluster gets a connection refused or a timeout. Your application logs show nothing at all, because no request ever arrived. The silence is the evidence.

Two things have to be true for an address to receive traffic: a pod must match the selector, and that pod must be Ready. Pods that are running but failing their readiness probe stay in the EndpointSlice marked ready: false and are deliberately left out of load balancing — that is the probe doing its job, not a fault in the Service.

One detail catches people repeatedly: matching is on labels, and labels live on the pod, not on the Deployment. A Deployment's own labels and the labels in its pod template are different fields, and a Service selecting the former finds nothing.

How to tell this is your problem
WhereWhat you see
kubectl get endpointslices -l kubernetes.io/service-name=<service>ENDPOINTS shows <unset>
An Ingress or gateway503, often worded as no healthy upstream
Your application logsNothing — the request never reached a pod
kubectl get podsPods may be Running and still show 0/1 READY

How to know it is actually fixed

  • kubectl get endpointslices -l kubernetes.io/service-name=<service> -o yaml lists at least one address with ready: true.
  • A request from inside the cluster to the Service name succeeds — test from a pod, not from your laptop.
  • Delete one pod and confirm the endpoint list drops it and then restores it. An endpoint list that is correct only while nothing changes is not yet correct.
  • If an Ingress was returning 503, confirm it now returns the application's own response rather than assuming it followed.

Stopping it happening again

  • Generate the Service selector from the same labels the Deployment template sets, rather than writing it twice.
  • Give every workload a readiness probe that checks something real, and confirm it fails when the application is genuinely unhealthy.
  • Use named ports so the Service and the container cannot drift apart numerically.
  • Add an endpoint-count check to deploy verification. A rollout that reports success against an empty Service is the failure this page exists for.

Security considerations

  • A Service selects by label alone, within its namespace. Anyone who can create a pod with matching labels there can receive the Service's traffic once that pod is ready, so keep selectors specific and limit who can create pods in production namespaces.
  • Do not set publishNotReadyAddresses: true to make addresses appear. It marks every matching pod as ready in the EndpointSlice, so traffic reaches pods that have not passed their readiness probe.
  • If the Debug Services steps lead you to a NetworkPolicy, add a narrow allow rule for the client and port instead of deleting the policy. A pod that no policy selects for ingress accepts connections from everywhere.
  • Testing from inside the cluster means running commands in a production pod with kubectl exec, or starting a test pod. Use an image your cluster allows, and let --rm delete the test pod when you exit, as the Debug Services command does.

Sources

Behaviour described here is drawn from official documentation. Where a figure could not be confirmed on an official page it is attributed in the text rather than stated as canonical.

Did this get you to an answer?

No text box on purpose — please do not paste production logs anywhere

Something wrong or out of date? Report it