Skip to main content

Kubernetes · troubleshooting

Pod stuck in Pending

The pod exists in the API but no node has accepted it. Nothing is running yet, so logs are empty — the answer is always in the scheduler's own explanation.

Run this first

step 1 of 3
kubectl describe pods <pod>

Look for FailedScheduling, and the count in "0/N nodes are available" with the reason for each

The scheduler writes its refusal here, per node. This single command usually contains the whole answer.

Work out which cause you have

A few questions to narrow the list. Every answer ends in a command that confirms or rules the cause out — this cannot see your cluster, so nothing here is a certainty until you have checked.

Narrow it down

0 answered · nothing is sent anywhere

What reason does the FailedScheduling event give?

In kubectl describe pod, look for the line beginning "0/N nodes are available".

Cause space

5 of 5 still possible

  • No node has enough free CPU or memoryCommon
  • Node selectors, affinity or taints exclude every nodeCommon
  • A PersistentVolumeClaim is unboundOccasional
  • A ResourceQuota is blocking creationOccasional
  • There are no schedulable nodesRare

Nothing ruled out yet. Answer the question above and the branches your answer eliminates will strike through here.

Check it with a tool

Or diagnose it manually

In this order. The first command usually contains the whole answer.

  1. Step 1

    kubectl describe pods <pod>

    The scheduler writes its refusal here, per node. This single command usually contains the whole answer.

    Look for FailedScheduling, and the count in "0/N nodes are available" with the reason for each

  2. Step 2

    kubectl get nodes

    Rules out the case where nothing is schedulable at all, which would make the rest of the investigation moot.

    Look for Ready status, and SchedulingDisabled on any cordoned node

  3. Step 3

    kubectl describe nodes | grep -A5 'Allocated resources'

    Shows committed requests rather than live usage — the numbers the scheduler actually reasons about.

    Look for CPU or memory requests already near 100% of allocatable

Every cause, and how to fix it

Ordered by how often each one turns out to be the answer.

No node has enough free CPU or memory

Common

The pod's *requests* — not its usage — exceed what any single node has unallocated. This catches people because a cluster can look busy at 40% actual utilisation while being fully committed on requests. The scheduler reasons about requests, not about what is really being used.

Confirm

kubectl describe pod <pod> | grep -A10 Events

"Insufficient cpu" or "Insufficient memory" in the FailedScheduling message

Fix

  • Lower the request if it was set higher than the workload genuinely needs.
  • Add capacity, or free it by right-sizing over-requested pods elsewhere.
  • Check the request is not accidentally enormous — a units mistake such as 1000 instead of 1000m asks for a thousand CPUs.
kubectl describe nodes | grep -A5 'Allocated resources'

Shows committed requests per node, which is what the scheduler actually weighs.

Node selectors, affinity or taints exclude every node

Common

The pod asks for nodes with a particular label, or the only suitable nodes carry a taint the pod does not tolerate. Capacity is irrelevant here — the scheduler has excluded those nodes from consideration entirely.

Confirm

kubectl describe pod <pod> | grep -i -E 'didn.t match|untolerated taint|node.selector|affinity'

"node(s) didn't match Pod's node affinity/selector" or "had untolerated taint"

Fix

  • Compare the pod's nodeSelector and affinity against real node labels: kubectl get nodes --show-labels.
  • Add the matching toleration, or remove the constraint if it was inherited from a copied manifest.
  • Control-plane nodes are tainted deliberately — tolerating that taint is rarely the right answer.

A PersistentVolumeClaim is unbound

Occasional

The pod mounts a PVC that has not been satisfied. No matching PersistentVolume exists, no StorageClass can provision one, or the volume is zone-bound to a zone with no suitable node. The pod waits on storage, not on compute.

Confirm

kubectl get pvc -n <namespace>

A PVC in Pending rather than Bound

Fix

  • Check the StorageClass named by the PVC exists and can provision dynamically.
  • For zonal volumes, confirm a node exists in the volume's zone.
  • Read the PVC's own events — they usually name the provisioning failure directly.

A ResourceQuota is blocking creation

Occasional

The namespace has a quota and this pod would exceed it. The failure is admission-level rather than scheduling, so the wording differs from the usual capacity messages — which makes it easy to misread as insufficient resources.

Confirm

kubectl describe quota -n <namespace>

Used values at or near Hard limits for the resource the pod requests

Fix

  • Free quota by removing unused workloads in the namespace, or raise the quota.
  • Confirm the pod sets the requests the quota requires — some quotas reject pods with no requests at all.

There are no schedulable nodes

Rare

Every node is NotReady, cordoned, or the cluster genuinely has none. Obvious once seen, and worth ruling out early because it makes every other investigation pointless.

Confirm

kubectl get nodes

All nodes NotReady or SchedulingDisabled, or an empty list

Fix

  • Recover or uncordon the nodes: kubectl uncordon <node>.
  • If a cluster autoscaler should have added capacity, check its logs — it may be at its own maximum.

Understanding it properly

Skip this if you are mid-incident — the working part of the page is above. Worth reading afterwards, because understanding the mechanism is what stops the next one.

What is actually happening

Pending means the pod has been created but not yet placed on a node. The Kubernetes documentation is direct about this: a Pending pod is one that cannot be scheduled. There is no container, so there are no logs to read and no exit code to interpret. Looking for either is the most common wasted first move.

What you do have is better. The scheduler records why it declined, per node, in the pod's events — usually as a line like "0/3 nodes are available" followed by the specific reason for each. That message is the diagnosis; everything else is confirming it.

One nuance worth knowing: a pod can also sit Pending while waiting on a volume rather than a node. If the events mention a PersistentVolumeClaim, the scheduler is not the thing blocking you.

How to tell this is your problem
WhereWhat you see
kubectl get podsSTATUS Pending, 0/1 READY, and it does not change
kubectl describe podFailedScheduling events, typically "0/N nodes are available"
kubectl logsNo output at all — there is no container yet

How to know it is actually fixed

  • kubectl get pod shows the pod has left Pending and reached Running or ContainerCreating.
  • kubectl describe pod shows a Scheduled event naming the node it landed on.
  • If a PVC was involved, it now reports Bound rather than Pending.

Stopping it happening again

  • Set requests from measurement so the scheduler has an accurate picture and the cluster is not falsely full.
  • Watch committed requests, not just utilisation — a cluster can be fully committed while looking idle.
  • Keep node labels and pod selectors under review; copied manifests carry constraints that no longer apply.
  • Confirm StorageClasses can actually provision in the zones your nodes occupy.

Did this get you to an answer?

No text box on purpose — please do not paste production logs anywhere

Sources

Behaviour described here is drawn from official documentation. Where a figure could not be confirmed on an official page it is attributed in the text rather than stated as canonical.

Related

Pod stuck in Pending — causes, diagnosis and fix | DevOps Insights