# 🔐 Kubernetes - Secrets

We’ve already seen how to use ConfigMaps to pass **non-sensitive data** like configs and files to Pods. If you missed that, you can check it out [here](https://claybrainer.com/kubernetes-configmaps).

But now comes the real question:

*   *What about sensitive data?*
    
*   *Passwords? Tokens? SSH keys?*
    

That’s where **Kubernetes Secrets** come in. In this blog, we’ll cover:

*   What are Secrets and why we need them
    
*   How to create Secrets
    
*   How Pods consume them
    
*   Lifecycle & behavior
    
*   Security best practices (this is where most people mess up 😅)
    

# 🤔 What are Kubernetes Secrets?

A **Secret** is a Kubernetes object used to store **sensitive data** like: *Passwords , API tokens, SSH keys , Certificates.*

Instead of hardcoding these inside your app or YAML, you **decouple them** using Secrets.

**⚙️ How are they different from ConfigMaps?**

They look similar to ConfigMaps however they have some distinct behaviour

*   **Stored in etcd** (cluster DB)
    
*   **Mounted as tmpfs (RAM)** → never written to disk 🧠
    
*   **Base64 encoded** (⚠️ not encryption!)
    

> 💡 Important: Base64 ≠ Security. Anyone with access can decode it easily.

# 😌 Why do we need Secrets?

You already know the answer, but let’s reinforce:

*   Keeps sensitive data out of code
    
*   Helps follow **12-factor app principles**
    
*   Enables secure configuration per environment
    

You can think of it like this

*   ConfigMap = app config
    
*   Secret = app credentials
    

# 🛠️ How to Create Secrets

You can create Secrets in **two ways**:

*   Inline Command
    
*   Using YAML
    

Let's explore them in detail

### ⚡ 1. Inline Command (Quick & Dirty)

**Example:**

```shell
# Basic syntax
kubectl create secret <type> <name> --from-literal=<key>=<value>

# Example
kubectl create secret generic api-credentials \
  --from-literal=token=super-secret-123
```

**Multi-value example:**

```shell
kubectl create secret docker-registry private-reg-auth \
  --docker-username=my-user \
  --docker-password=my-password \
  --docker-server=my-registry.io \
  --docker-email=my-email@example.com
```

## When to Choose this approach ?

Use this for:

*   Quick testing
    
*   PoCs
    

> Note: Here you can directly pass the values without encoding, Kubernetes encode the value and store it in Etcd

## 📄 2. Using YAML (Production Way)

```yaml
apiVersion: v1
kind: Secret
metadata:
  name: my-db-secret
type: Opaque
data:
  username: YWRtaW4=
  password: MWYyZDNoNGo1aw==
```

Here you **must encode values manually** using `echo -n "admin" | base64`

## 🧩 Secret Types (Quick Understanding)

| Type | Use Case |
| --- | --- |
| `Opaque` | Default, anything |
| `kubernetes.io/dockerconfigjson` | Private registry auth |
| `kubernetes.io/tls` | TLS cert + key |
| `kubernetes.io/basic-auth` | Username/password |
| `kubernetes.io/ssh-auth` | SSH keys |
| `service-account-token` | Internal API access |

> 💡 Tip: Use specific types when possible — Kubernetes validates structure.

# 🔌 How Pods Consume Secrets

Two main ways (same as ConfigMaps 👀):

*   Environment Variables
    
*   Volume Mounts
    

## 🛡️ Option 1: Environment Variables

This option is suitbale for for API keys or database passwords that the application expects to find in its environment. This is simple and clean approach

**Example**

```yaml
apiVersion: v1
kind: Pod
metadata:
  name: my-app-pod
spec:
  containers:
    - name: my-app
      image: nginx
      env:
        - name: DB_PASSWORD
          valueFrom:
            secretKeyRef:
              name: db-creds
              key: password
```

⚠️ Cons:

*   Values are loaded **only at startup**
    
*   If Secret changes → **Pod restart required** 🔄
    

### 📁 Option 2: Volume Mounts (Recommended)

This option is best for certificates, SSH keys, or large configuration files. Kubernetes mounts the secret as a directory where each key in the secret becomes a file.

**Example:**

```yaml
apiVersion: v1
kind: Pod
metadata:
  name: secret-volume-pod
spec:
    containers:
    - name: ssh-container
      image: nginx 
      volumeMounts:
        - name: ssh-key-vol 
          mountPath: "/etc/ssh-keys" 
          readOnly: true 
volumes: 
    - name: ssh-key-vol 
      secret: 
        secretName: ssh-key-secret # The name of the Secret object
```

👉 Each key becomes a file:

```plaintext
/etc/ssh-keys/
  ├── private_key
  ├── public_key
```

### 💡 Why use Volumes over Environment Variables?

*   **Security**: Secret volumes are stored in **tmpfs** (RAM). If the node is powered down, the data vanishes from the node's memory. 🧠
    
*   **Updates**: If you update a Secret, Kubernetes automatically updates the files in the volume (though this takes a few seconds). Environment variables, however, are static—you would have to restart the Pod to see a change. 🔄
    
*   **Structure**: Volumes are perfect for things that naturally exist as files, like SSL certificates or SSH `id_rsa` keys.
    

### 📁 Standard Mount vs. subPath

## 🔹 Standard Mount (Default Behavior)

When you mount a Secret as a volume:

*   The **entire target directory gets replaced**
    
*   Any existing files become **invisible**
    
*   Only the **mounted Secret content is available for your pod to use.**
    

👉 If you mount another Secret to the same path, it will **overwrite the previous one**

This behavior is intentional in Kubernetes to Ensure **data isolation,** Improve **security,** Avoid unintended file conflicts

Now consider a scenario where you want to mount **multiple Secrets into the same Pod path**.

Using the standard approach:

*   Mounting **Secret A** → works fine
    
*   Mounting **Secret B to the same path** → **overwrites Secret A**
    

👉 Result: The first Secret becomes inaccessible.

To solve this, Kubernetes provides a feature called `subPath`.

### 💡 What is `subPath`?

`subPath` allows you to:

*   Mount **individual files or subdirectories**
    
*   Place them **inside an existing directory**
    
*   Avoid overriding the entire volume path
    

> To know in detail about subpath [check here](https://claybrainer.com/kubernetes-configmaps#subpath)

✅ Using `subPath` for Multiple Secrets

With `subPath`, you can mount multiple Secrets into the same directory **without conflicts**.

**Example:**

```yaml
volumeMounts:
  - name: secret-a
    mountPath: /etc/config/secret-a.txt
    subPath: secret-a.txt
  - name: secret-b
    mountPath: /etc/config/secret-b.txt
    subPath: secret-b.txt
volumes:
  - name: secret-a
    secret:
      secretName: secret-a
  - name: secret-b
    secret:
      secretName: secret-b
```

### 🔍 What happens here?

*   `secret-a` is mounted as `/etc/config/secret-a.txt`
    
*   `secret-b` is mounted as `/etc/config/secret-b.txt`
    
*   No overwriting occurs
    
*   Both Secrets coexist in the same directory 🎉
    

One thing to note here is that when using `subPath` we have to define path with filename and extension whereas in standardMount the mount path is sufficient enough.

> LifeCycle for `subPath`  
> One more important thing is that when using `subPath` when the secret value is updated it will not update the file directly pod restart is required because here the file is a static bind-mount

# 🔄 Lifecycle of Secrets

In this section we will explore how changing a secret value will be handled in different Consumption option

| Consumption Type | Behavior |
| --- | --- |
| Env Variables | Deployment Requires restart 🔁 |
| Volume Mount | Auto updates ⏱️ |
| subPath | Deployment Requires restart 🔁 |

> Even though VolumeMount Auto Updates your secrets to the Mount file you app should be configured to pick the updated value.

# Security Best Practices & RBAC

Now that we know how to create and consume Secrets, Let's talk about how to actually keep them "secret."

Even though Secrets are better than plain-text ConfigMaps, they have some vulnerabilities by default:

*   **Base64 is not encryption**: It's just a way to store binary data as text. Anyone who can run `kubectl get secret -o yaml` can see your passwords. 🔓
    
*   **Etcd Storage**: By default, Secrets are stored in plain text in the cluster's database (`etcd`).
    

## Some of the best practices

### Encryption at Rest (The Native Way)

You can configure the Kubernetes API server with an EncryptionConfiguration file. When this is enabled, the API server will mathematically encrypt the Secret data before it saves it to etcd. Even if someone steals the etcd hard drive, the data is unreadable garbage without the encryption key.

### Strict RBAC (Role-Based Access Control)

As we discussed briefly before, you must restrict who can run kubectl get secrets. If a developer has the get or list permission for Secrets, they can just ask the API server to show them the Base64 data, bypassing all etcd encryption. You should only grant Secret access to the specific ServiceAccounts that the Pods use, not to human users.

To truly secure a Secret while still letting it be used:

*   For Users (Developers): You generally want to deny get, list, and watch. If they can't get it, they can't see the password.
    
*   For Pods: The Pod doesn't actually need the user to have permissions. The Pod uses a ServiceAccount. The Kubernetes controller-manager handles the mounting process, so as long as the Pod's YAML is allowed to be created, the system handles the rest.
    

### External Secret Stores (The Enterprise Way)

In large companies, they don't even store the source-of-truth secrets in Kubernetes at all! They use tools like HashiCorp Vault, AWS Secrets Manager, or Azure Key Vault. They use plugins (like the Secrets Store CSI Driver or External Secrets Operator) to fetch the password from the cloud vault and inject it directly into the Pod at runtime.  

# Conclusion

Kubernetes Secrets might look simple at first, but how you use them makes all the difference. They help you keep sensitive data out of your code and configs—but they are **not secure by default**.

If you use them the right way prefer volumes over env vars, restrict access with RBAC, and integrate with external secret managers—you move from just “working setup” to a **production-ready secure system** 🔐

Thanks for reading. Keep learning, keep building, and keep breaking things (that’s how we grow) . See you in the next one ✌️