Kubernetes Secrets Are Not Secret

The Name Is the Problem

Kubernetes calls the object a Secret, so people assume the platform
keeps it secret. It does not, not by default, and the assumption is
exactly where the trouble starts. A Secret is a slightly more
careful ConfigMap. It is not a vault, it is not encrypted, and
treating it as if it were is one of the most common security gaps I
find in a cluster.

The core fact is simple. A Secret value is stored base64-encoded.
Base64 is not encryption. It is a way to represent bytes as text, and
it is trivially reversible by anyone, with no key and no permission.
Anybody who can read the Secret object can turn that string back into
the plaintext password in one step. The encoding changes what the
value looks like. It changes nothing about who can read it.

Where the Value Actually Lives

When you create a Secret, the value travels to the API server and
lands in etcd, the key-value store behind the control plane. Unless
you explicitly turned on encryption at rest, it sits in etcd as plain
base64. That is the same as plaintext for anyone who reaches the
store.

This matters more than it sounds, because etcd is a bigger surface
than people picture. An etcd backup or snapshot is a full copy of
every Secret in the cluster. If those backups land in object storage,
on a backup server, or in a disaster-recovery bucket without their
own encryption, then every credential in the cluster is sitting in
that file readable. A leaked etcd snapshot is a leak of everything at
once. Plenty of real breaches are exactly this: not a clever exploit,
just a backup nobody encrypted.

Who Can Read Them

Base64 in etcd is the storage problem. The access problem is worse,
because the number of paths to a Secret is larger than most teams
account for.

Anyone with etcd access reads them directly from the store. Anyone
with get on Secrets through RBAC reads them through the API, and
get secrets handed out too broadly is one of the most common
overprivileged-RBAC findings there is. Anyone who can create a pod in
the namespace can mount the Secret into that pod and read it from
inside, which means pod-create is effectively secret-read for every
Secret in the namespace. Anyone who can read your deployment
manifests, in Git, in CI logs, in a pipeline artifact, reads whatever
was written there.

That last one is the quiet killer. A Secret manifest committed to Git
is plaintext in your history forever, and Git never forgets. The
credential can be rotated a dozen times and the old one still sits in
the commit log for anyone who clones the repo. GitOps makes this
worse by encouraging you to store the manifest that defines the
Secret, which is the plaintext, in the repository by design.

None of these paths require breaking anything. They are all normal,
sanctioned access that happens to also grant secret-read as a side
effect nobody intended.

The Decode Leaves No Trace

There is a second sharp edge. Reading a Secret and decoding it is not
a privileged operation with its own audit trail. It is a normal API
read followed by a local decode on the reader's machine. The decode
step touches nothing on the cluster. There is no log line that says
"someone turned this Secret into a plaintext password," because the
part where it became plaintext happened on a laptop, not on the
server. You get, at most, a record that the object was read, and only
if API audit logging is on. The actual exposure is invisible.

What It Is Actually For

None of this makes Secrets useless. It makes them a different tool
than the name implies. A Secret is the right way to get a credential
out of your container image and out of your ConfigMaps, to mount it
into a pod as a file or an env var, and to update it without
rebuilding the image. That is real value. It is decoupling and
delivery, not confidentiality.

The mistake is stopping there and believing the confidentiality was
included. It was not. The object delivers the credential to the pod.
Keeping the credential actually secret is a separate set of decisions
you have to make on purpose.

What Actually Makes Them Secret

Three layers, in order of how much they buy you.

Turn on encryption at rest. This is the thirty-minute fix, and it is
the one nobody does until an auditor or an incident forces it. With an
EncryptionConfiguration on the API server, ideally backed by a real
KMS, the values in etcd stop being plaintext, and a leaked etcd
snapshot stops being a leak of everything. This is the single highest
return change available.

Lock down RBAC. Treat get, list, and watch on Secrets as dangerous
verbs. Grant them to the few components and humans that genuinely need
them, per namespace, and to nobody else by default. Remember that
pod-create in a namespace is secret-read in that namespace, and scope
accordingly.

Keep plaintext out of Git. If you do GitOps, the Secret has to be
encrypted before it is committed, with something like Sealed Secrets
or SOPS, so what lives in the repository is ciphertext and the key
lives somewhere the repository cannot reach. For anything past a
small cluster, an external secret store with real rotation and its
own audit log is where this ends up, with the Kubernetes Secret
becoming a short-lived injection point rather than the system of
record.

The Point

The word Secret promises confidentiality that the object does not
provide. Out of the box it is base64 in etcd, reachable through more
paths than you are tracking, decodable with no trace. That is fine as
long as you know it, and dangerous the moment you assume the platform
handled the hard part for you.

Encrypt etcd, keep the RBAC verbs tight, and never let plaintext
reach Git. The Secret is a delivery mechanism. The secrecy is your
job.