Skip to main content
DevOps Insightsguides & answers

Kubernetes · commonly confused

ConfigMap vs Secret

Short answer

A Secret is not encrypted by default. Kubernetes stores Secrets unencrypted in etcd unless encryption at rest is turned on, and base64 is only the format the data field has to be written in. What a Secret does give you is narrower handling — it is sent only to nodes that need it and, when mounted as a volume, kept off disk there — so treat it as a label for sensitive data, not as protection by itself.

First published

On this page · 10 sections

In simple terms

Both hold configuration that a pod reads at runtime, as environment variables or as files. A ConfigMap is for ordinary settings — a log level, a feature flag, a config file. A Secret is for things that would hurt if they leaked — passwords, tokens, keys. The YAML looks almost identical, which is exactly why people assume the difference is bigger than it is.

The belief worth correcting is that putting something in a Secret encrypts it. It does not. The Kubernetes documentation states plainly that Secrets are, by default, stored unencrypted in the API server's data store, etcd. The values look scrambled in YAML because the data field has to be base64-encoded, but base64 is a way of writing bytes as text, not a lock — anyone can decode it in one command.

So who can read a Secret? More people than most teams expect. The documentation lists them: anyone with API access to it, anyone with access to etcd, and — the one that surprises people — anyone allowed to create a Pod in that namespace, because they can simply start a pod that mounts the Secret and prints it. Permission to deploy is, in practice, permission to read the namespace's Secrets.

What a Secret genuinely buys you is careful handling, not secrecy. It is only sent to a node if a pod there needs it, and when it is mounted as a volume, the kubelet keeps it in memory-backed storage so it is not written to the node's disk. Those are real protections, and they are the reason to use a Secret rather than a ConfigMap for credentials — but they protect the copy on the node, not the copy in the cluster's database.

Words used on this page

ConfigMap
An API object holding non-confidential settings as key-value pairs, which Pods read as environment variables, command arguments or files.
Secret
An API object for confidential values such as passwords, tokens and keys. Pods use it the way they use a ConfigMap, and nodes handle it more carefully.
base64
A way of writing any bytes as plain text characters. Anyone can turn it back without a key: it is an encoding, not encryption.
etcd
The key-value database where the API server stores every object, Secrets included.
encryption at rest
An API server setting that encrypts chosen resources before they are written to etcd. It is off by default, and reads through the API still return plain values.
RBAC
Role-based access control: Roles and RoleBindings that say who may get, list, watch or create which kinds of object.
ServiceAccount
An identity for the processes in a Pod. A Pod runs as one, and can do whatever RBAC allows that account.
tmpfs
A Linux filesystem kept in memory. The kubelet uses it for Secret volumes so the values are not written to the node's disk.
stringData
A Secret field that accepts plain strings. The API server merges them into data, where they are stored as base64 like the rest.
external secret store
A system outside the cluster that holds the real values. Pods reach it through a driver such as the Secrets Store CSI Driver.

What actually happens

The documentation is explicit on the default: Kubernetes Secrets are stored unencrypted in the API server's underlying data store, etcd. Anyone with API access can retrieve or modify a Secret, and so can anyone with access to etcd. Anyone authorized to create a Pod in a namespace can read any Secret in that namespace, including indirectly — the documentation names the ability to create a Deployment as enough.

The values for all keys in a Secret's data field have to be base64-encoded strings; the stringData field accepts plain strings instead and is converted on write. That is a serialization rule for binary-safe storage, and it is identical in effect to plain text for anyone who can read the object. A ConfigMap's own documentation is equally direct: it does not provide secrecy or encryption.

The handling differences are where a Secret earns its name. A Secret is only sent to a node if a pod on that node requires it. When it is mounted into a pod, the kubelet stores a copy in a tmpfs so the data is not written to durable storage, and deletes that local copy once the pod that depends on it is gone. Both objects are capped at 1 MiB, which keeps large payloads from exhausting the API server and kubelet memory.

To make Secrets actually confidential, the documentation lists the minimum steps: enable encryption at rest for Secrets, configure RBAC with least-privilege access to them, restrict Secret access to specific containers, and consider an external secret store provider. Without them, a Secret's protection is mostly in how the node handles it. The default roles do treat the two differently (the built-in view role cannot read Secrets), but any role that grants list or watch on Secrets can read every one in its namespace, and get reads any Secret whose name is known unless the rule names specific Secrets with resourceNames.

What the base64 in a Secret is actually worth

# A Secret, as most people write it.
apiVersion: v1
kind: Secret
metadata:
  name: db-credentials
type: Opaque
data:
  password: czNjcjN0LXBhc3N3MHJk   # looks protected

# Anyone who can read the object reverses it in one step:
kubectl get secret db-credentials -o jsonpath='{.data.password}' | base64 --decode
# s3cr3t-passw0rd

# Two checks that matter more than the YAML.
# Who in this namespace can read Secrets directly?
kubectl auth can-i get secrets --as=system:serviceaccount:<namespace>:default -n <namespace>

# Can this identity create pods here? If so, it can mount any Secret
# in the namespace and print it, whatever the RBAC on Secrets says.
kubectl auth can-i create pods --as=<user> -n <namespace>

The password is a placeholder. Whether the value is encrypted where it is stored depends on whether encryption at rest has been configured for the cluster, which the Secret object itself cannot tell you.

Try it yourself

Before you start

  • A test cluster and kubectl, and a namespace you can throw away. Use the example's made-up password, never a real one: these commands print it on your screen, and the decode step can leave it in your shell history.
  • Permission to create and read Secrets there: kubectl auth can-i create secrets -n <namespace> and kubectl auth can-i get secrets -n <namespace> should both print yes. The built-in view role is not enough, because it cannot read Secrets. Checking another identity with --as also needs permission to impersonate.
  • base64 on your own machine. The decode below gives the same result with the base64 that ships with macOS; Linux has one in coreutils.
  • Single quotes around any value containing $, \, *, = or !. The Kubernetes page warns that your shell will otherwise interpret them: unquoted in a bash script, S!B\*d$zDsb= became S!B*d=.
  • To find out whether Secrets are encrypted in etcd: a look at the kube-apiserver's flags. With no --encryption-provider-config, or with identity as the first provider in that file, they are stored unencrypted. On a managed cluster, your provider's documentation says how to check.

Creating a Secret and reading it back

Quoted from the Kubernetes task page Managing Secrets using kubectl; not reproduced here, because no cluster was available. The page labels each output as similar to what you will see. Its create command is split over three lines with backslashes; it is joined onto one line here. The password is the page's own example value.

Command

kubectl create secret generic db-user-pass --from-literal=username=admin --from-literal=password='S!B\*d$zDsb='
kubectl describe secret db-user-pass
kubectl get secret db-user-pass -o jsonpath='{.data}'
echo 'UyFCXCpkJHpEc2I9' | base64 --decode

Output · as shown in Kubernetes: Managing Secrets using kubectl

$ kubectl create secret generic db-user-pass --from-literal=username=admin --from-literal=password='S!B\*d$zDsb='
secret/db-user-pass created

$ kubectl describe secret db-user-pass
Name:            db-user-pass
Namespace:       default
Labels:          <none>
Annotations:     <none>
Type:            Opaque
Data
====
password:    12 bytes
username:    5 bytes

$ kubectl get secret db-user-pass -o jsonpath='{.data}'
{ "password": "UyFCXCpkJHpEc2I9", "username": "YWRtaW4=" }

$ echo 'UyFCXCpkJHpEc2I9' | base64 --decode
S!B\*d$zDsb=

How to read it

  • secret/db-user-pass created: kubectl took the values as you typed them and did the base64 encoding itself. The single quotes kept the shell from rewriting the password.
  • kubectl describe lists only key names and sizes, such as password: 12 bytes. The page explains that get and describe hide values by default so they do not land on screen or in a terminal log. That is a display choice, not access control.
  • -o jsonpath='{.data}' prints the stored values. They look scrambled, but they are base64, and they are exactly what anyone allowed to get this Secret receives.
  • base64 --decode turns UyFCXCpkJHpEc2I9 back into S!B\*d$zDsb= with no key at all. Encryption at rest would not change this: the API server decrypts before it answers.
  • The page warns that typing the encoded value into echo leaves it in your shell history, and suggests one pipeline instead: kubectl get secret db-user-pass -o jsonpath='{.data.password}' | base64 --decode.

The mistake this causes

Treating a Secret as encrypted, and granting broad pod-creation or read access on that assumption.

Why people do it The name and the base64 encoding both suggest protection. Neither provides it: Secrets are unencrypted in etcd by default, base64 is reversible by anyone, and anyone who can create a pod in the namespace can mount a Secret and read it.

What you see Nothing fails, which is the problem. Credentials end up readable by every CI service account and developer with deploy rights to the namespace, by anything that can read etcd or its backups, and in any YAML that gets committed with the data block intact. It shows up later, as a leaked credential with no obvious path to it.

If it goes wrong

The Pod stays in ContainerCreating with the event MountVolume.SetUp failed for volume "<volume>" : secret "<name>" not found
The kubelet looks the Secret up in the Pod's own namespace, and it is not there. Check with kubectl get secret <name> -n <namespace>, and create it in that namespace or fix the name. If the Pod can genuinely run without it, mark the volume's Secret optional: true.
CreateContainerConfigError, with the event Error: couldn't find key password in Secret <namespace>/<name>
The Secret exists but has no key by the name your secretKeyRef uses. kubectl describe secret <name> lists the keys it does have. Fix the key in the Pod spec or the Secret.
The application rejects a password that looks right
If you created the Secret from a file, the file may end with a newline, and kubectl encodes that too. Recreate the file with echo -n, or use --from-literal. kubectl get secret <name> -o jsonpath='{.data.password}' | base64 --decode | od -c shows a trailing \n if there is one.
The decoded value is shorter than what you typed, for example everything from a $ onwards is missing
The shell rewrote the value before kubectl saw it. Wrap it in single quotes, as the Kubernetes page says, or put it in a file, where special characters need no escaping.
Creating the Secret fails with data: Too long: may not be more than 1048576 bytes
A Secret's data is capped at 1 MiB, and ConfigMaps have the same cap. Split the data across several objects, or keep large files in a volume or an external store and put only credentials in the Secret.

How it shows up in production

A team moves database credentials from a ConfigMap into a Secret after a review flags them, and marks the finding closed. The pipeline's service account keeps the broad role it always had, because the Secret is assumed to be safe now.

Months later a dependency in an unrelated job is compromised. The job runs under that service account, which can create pods in the namespace. The attacker starts a pod that mounts the Secret and prints it — no exploit needed, just the permissions the job already had.

The post-incident question is how the credential leaked from an encrypted store. The cluster did not have encryption at rest configured, but that is not the answer: a Pod that mounts a Secret receives the plain value either way. The RBAC that mattered was not on the Secret at all but on pod creation.

The fix that would have prevented it is in the documentation's own list: least-privilege RBAC that treats create-pods as read-secrets, and scoping the job's identity to only what it needed. Encryption at rest belongs on the list too, but it protects etcd and its backups, not a Secret read through a Pod.

How to tell which one you are hitting

Is this value sensitive if it leaks?
If yes, use a Secret — for the narrower handling on nodes, and so it is covered by whatever encryption and RBAC you apply to Secrets. If no, a ConfigMap is fine, and keeping non-sensitive data out of Secrets makes the sensitive set easier to audit.
Is encryption at rest actually configured for Secrets on this cluster?
Check, do not assume. Managed platforms differ, and nothing on the Secret object says either way. Without it, Secrets are stored unencrypted in etcd and anything that reads etcd or its backups reads them.
Who can create pods in this namespace?
That list is, in effect, the list of who can read its Secrets, because a pod can mount any Secret in its namespace. Review it alongside the RBAC on Secrets themselves, not separately.
Would an external secret store fit better?
The documentation suggests considering one. It keeps the source of truth outside the cluster. Rotation and audit then depend on the store you choose; for plain Secrets, the Kubernetes docs suggest short-lived Secrets and audit rules that alert on unusual reads.

Security considerations

  • Encryption at rest protects etcd and its backups, not the API. With it on, kubectl get secret and any Pod that mounts the Secret still receive the plain value, so it does nothing against someone with RBAC access or the right to create Pods. It also only covers Secrets written after you enable it: the docs have you rewrite the existing ones with kubectl get secrets --all-namespaces -o json | kubectl replace -f -.
  • list or watch on Secrets returns every Secret's contents in the namespace, not just names. The built-in edit role can read Secrets and run Pods as any ServiceAccount in the namespace. Give people and CI jobs the narrowest role, and limit get to named Secrets with resourceNames where you can.
  • A container running with privileged: true can read every Secret used on its node, whichever Pod it belongs to. The Baseline Pod Security Standard disallows privileged Pods; enforce it on namespaces that do not need them, so one compromised workload cannot collect its neighbours' credentials.
  • A manifest with a data block holds the real credential. Committing it to Git or pasting it into a ticket publishes the value to everyone who can read it, because base64 adds no confidentiality. Keep such manifests out of source control and rotate anything that leaked.
  • The tmpfs protection is for Secret volumes. On nodes with swap enabled, the kubelet mounts them with the noswap option only if the kernel supports it (Linux 6.3, or a backport); otherwise it logs a warning, and the values could be swapped to disk.

In an interview

How this gets asked in an interview

Usually phrased as What is the difference between a ConfigMap and a Secret, and is a Secret secure?

What a strong answer contains The strong answer corrects the premise: a Secret is not encrypted by default — it is stored unencrypted in etcd unless encryption at rest is enabled, and base64 is just the encoding the data field requires. Its real advantages are handling: it is only sent to nodes that need it and, when mounted as a volume, kept in tmpfs there rather than on disk. Making Secrets confidential needs encryption at rest plus least-privilege RBAC — and recognizing that permission to create pods in a namespace is permission to read its Secrets.

The follow-up "A service account can't get secrets, but can create deployments. Can it read the database password?" Yes — it can create a pod that mounts the Secret and print it. The documentation calls out creating a Deployment as exactly this kind of indirect access.

Sources

The behaviour this page describes, as documented by the projects themselves.

Did this get you to an answer?

No text box on purpose — please do not paste production logs anywhere

Something wrong or out of date? Report it