Skip to main content
DevOps Insightsguides & answers

Kubernetes · troubleshooting

error: You must be logged in to the server (Unauthorized)

The API server was reached but rejected the credentials kubectl sent, with HTTP 401. Usually the client certificate has expired (kubeadm's last one year), a token or cloud login has expired or maps to no one, or the context uses the wrong user. Find which credential the context uses, check it, and renew or replace it.

Reproduced on kubectl 1.37.1, . Where the text says “reproduced”, the behaviour was run on that version, not taken from documentation.

First published

New to this? Read what is actually happening first, then come back to the steps.

On this page · 9 sections

Before you start

  • Access to whatever issues your credential: the control-plane node for a kubeadm cluster, your cloud account for EKS, GKE or AKS, or the cluster administrator.
  • openssl, to read a certificate's expiry date.
  • For kubeadm, root on every control-plane node, since certificates are renewed on each.

Run this first

step 1 of 3
kubectl config view --minify

Look for client-certificate-data (cause one), an exec or token entry (cause two), and whether the user fits the cluster (cause three)

Shows the cluster and user in effect, with secrets redacted, so you know which credential type you are dealing with.

Work out which cause you have

A few questions to narrow the list. Every answer ends in a command that confirms or rules the cause out — this cannot see your cluster, so nothing here is a certainty until you have checked.

Narrow it down

0 answered · nothing is sent anywhere

What does the current context's user use to authenticate?

kubectl config view --minify

Cause space

3 of 3 still possible

  • The client certificate has expiredCommon
  • The token or cloud identity is expired or unknown to the clusterCommon
  • The context uses the wrong user for this clusterOccasional

Nothing ruled out yet. Answer the question above and the branches your answer eliminates will strike through here.

Or diagnose it manually

In this order. The first command usually contains the whole answer.

  1. Step 1

    kubectl config view --minify

    Shows the cluster and user in effect, with secrets redacted, so you know which credential type you are dealing with.

    Look for client-certificate-data (cause one), an exec or token entry (cause two), and whether the user fits the cluster (cause three)

  2. Step 2

    kubectl get --raw /api -v=6 2>&1 | grep -m1 -E 'Response|status'

    Confirms the server answers 401, as opposed to a connection or TLS failure.

    Look for 401 Unauthorized

  3. Step 3

    kubectl config view --raw --minify -o jsonpath='{.users[0].user.client-certificate-data}' | base64 -d | openssl x509 -noout -enddate

    Reads the certificate's expiry when the user is certificate-based. Prints nothing useful for token users.

    Look for A notAfter date in the past

Every cause, and how to fix it

Ordered by how often each one turns out to be the answer.

The client certificate has expired

Common

The kubeconfig authenticates with client-certificate-data and that certificate's notAfter date has passed. On kubeadm clusters this happens one year after init or the last upgrade, for admin.conf and every copy of it in people's ~/.kube/config.

Confirm

kubectl config view --raw --minify -o jsonpath='{.users[0].user.client-certificate-data}' | base64 -d | openssl x509 -noout -enddate

A notAfter date in the past. On a kubeadm control-plane node, sudo kubeadm certs check-expiration shows every certificate's remaining time

Fix

  • On each control-plane node, renew with kubeadm certs renew all.
  • Restart the control-plane static pods so they load the new certificates. kubeadm's documentation says to do this because not every component reloads certificates dynamically; move a manifest out of /etc/kubernetes/manifests/, wait about 20 seconds, then move it back.
  • Copy the renewed admin.conf into ~/.kube/config again, for everyone who had a copy.
  • Plan an upgrade or a renewal reminder well before next year's date.
sudo kubeadm certs check-expiration
sudo kubeadm certs renew all
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config

From kubeadm's certificate management guide. Run the renewal on every control-plane node, and restart the control-plane static pods in between, as that guide describes.

The token or cloud identity is expired or unknown to the cluster

Common

The user uses a token or an exec plugin. The cloud login behind the plugin has expired, the token was revoked, or, on EKS, the IAM principal has no access entry or is not the one that created the cluster.

Confirm

kubectl config view --minify -o jsonpath='{.users[0].user}'

An exec block names the plugin command; run it yourself to see its error. A token field means a static token, which may have been rotated or deleted

Fix

  • Log in again with the cloud CLI, then retry, for example aws sso login, gcloud auth login or az login.
  • Regenerate the kubeconfig with the provider's command, which writes the correct exec block for the current tools.
  • On EKS, check which IAM identity you are using with aws sts get-caller-identity, and have an administrator add an access entry for it.
aws sts get-caller-identity
aws eks update-kubeconfig --name <cluster> --region <region>

EKS shown. For GKE use gcloud container clusters get-credentials; for AKS, az aks get-credentials.

The context uses the wrong user for this cluster

Occasional

Several clusters in one kubeconfig, a merged KUBECONFIG, or a copied context can pair a cluster with a user meant for another cluster. The credential is valid somewhere, just not here.

Confirm

kubectl config get-contexts && kubectl config view --minify

The current context's cluster and user, and whether that user belongs to that cluster

Fix

  • Switch to the right context with kubectl config use-context <name>.
  • Or fix the context's user with kubectl config set-context <name> --user=<user>.

Understanding it properly

Skip this if you are mid-incident — the working part of the page is above. Worth reading afterwards, because understanding the mechanism is what stops the next one.

In plain terms

kubectl reached the cluster, but the cluster did not accept who kubectl said you were. Every request carries a credential, a certificate, a token or a login from a cloud tool, and this one was expired, wrong or unknown to the cluster. It is not a permissions problem; those say forbidden. The cluster does not know who you are at all.

A rejected credential, and checking a certificate's expiry

Reproduced with kubectl 1.37.1 on macOS against a minimal local stand-in that answers every request the way an API server answers bad credentials: HTTP 401 with Kubernetes' standard Status body, reason Unauthorized. The certificate check used a throwaway self-signed certificate with kubeadm's admin subject.

Command

kubectl get pods
kubectl config view --raw --minify -o jsonpath='{.users[0].user.client-certificate-data}' | base64 -d | openssl x509 -noout -subject -enddate

Output

$ kubectl get pods
E1018 … "Couldn't get current server API group list" err="the server has asked for the client to provide credentials"
…
error: You must be logged in to the server (the server has asked for the client to provide credentials)

$ kubectl config view --raw --minify -o jsonpath='…client-certificate-data}' | base64 -d | openssl x509 -noout -subject -enddate
subject=O=kubeadm:cluster-admins, CN=kubernetes-admin
notAfter=<date> GMT

How to read it

  • kubectl 1.37 puts the client library's wording in the brackets: the server has asked for the client to provide credentials. Other versions and other requests show (Unauthorized), which is how AWS's EKS troubleshooting page quotes it. Both mean the server answered 401.
  • The E lines come first because kubectl looks up the server's API list before running the command, and that lookup is what failed.
  • notAfter is the certificate's expiry. If it is in the past, that is the cause. The subject shows whose certificate it is: kubeadm's admin certificate has O=kubeadm:cluster-admins, CN=kubernetes-admin.

Words used on this page

Authentication
Proving who you are. A 401 Unauthorized means it failed. It comes before authorization, which decides what you may do; that failure is 403 Forbidden.
Client certificate
A certificate in the kubeconfig (client-certificate-data) that identifies you. It stops working when it expires.
Bearer token
A string in the kubeconfig, or produced by a plugin, that identifies you. It can expire or be revoked.
exec credential plugin
A command kubectl runs to get a token, such as aws eks get-token or gke-gcloud-auth-plugin. It fails or returns a rejected token when your cloud login has expired.
admin.conf
The administrator kubeconfig kubeadm writes. Its client certificate expires after one year like kubeadm's other client certificates.
What is actually happening

Every request kubectl sends carries a credential from the current context's user: a client certificate, a static token, or a token from an exec plugin. The API server tries its authenticators on it. If none accepts it, the server answers 401 Unauthorized and kubectl prints this error.

That makes it different from Forbidden. 403 means the server knows who you are and RBAC refused the action; 401 means it does not accept who you claim to be, so no permission change can fix it.

On kubeadm clusters the classic cause is time. Client certificates generated by kubeadm expire after one year, admin.conf included. kubeadm renews them during control-plane upgrades, so a cluster upgraded at least yearly never notices; one left alone for a year stops answering kubectl on the anniversary.

On managed clusters it is usually identity. AWS's troubleshooting guide lists this error when the IAM principal kubectl uses is not the one that created the cluster, or has no EKS access entry. GKE and AKS behave similarly when the cloud login behind the plugin has expired.

How to tell this is your problem
WhereWhat you see
kubectl output, recent versionserror: You must be logged in to the server (the server has asked for the client to provide credentials)
kubectl output, other versions and requestserror: You must be logged in to the server (Unauthorized)
kubectl -v=6Response status 401 Unauthorized
openssl x509 -enddate on the kubeconfig's client certificateA notAfter date in the past
EKS, before the errorcould not get token: AccessDenied: Access denied

How to know it is actually fixed

  • kubectl get nodes works.
  • kubectl auth whoami shows the identity you expect (Kubernetes 1.28 and later).
  • On kubeadm, sudo kubeadm certs check-expiration shows a residual time of about a year.

Stopping it happening again

  • Upgrade kubeadm clusters at least once a year, which renews certificates, or put kubeadm certs check-expiration into monitoring.
  • Use short-lived, per-person credentials from your identity provider or cloud instead of copies of admin.conf.
  • Keep one kubeconfig per cluster, or name contexts clearly, so users and clusters are not mixed up.

Security considerations

  • Renew or reissue credentials; do not work around this by switching to a more privileged kubeconfig you happen to have. admin.conf is cluster-admin, and kubeadm's documentation says not to share it.
  • Never paste a kubeconfig or a token into a ticket or chat while debugging. Use kubectl config view --minify, which redacts secrets unless you add --raw.
  • On EKS, add people through access entries for their own IAM identity rather than sharing the identity that created the cluster.
  • Treat certificate expiry as a scheduled event: an expired control-plane certificate can stop more than kubectl, so renew before the date rather than after the outage.

Sources

Behaviour described here is drawn from official documentation. Where a figure could not be confirmed on an official page it is attributed in the text rather than stated as canonical.

Did this get you to an answer?

No text box on purpose — please do not paste production logs anywhere

Something wrong or out of date? Report it

Where to go next