# Kubernetes - Volumes - Part-2 ## 👋 The "Node-Lock" Problem That "node-lock" is the biggest weakness of `hostPath`. If the node goes down or the Pod moves, the data is essentially "trapped" on the old node. To solve this, Kubernetes uses a system that decouples the storage from the node entirely, often using network storage (like a cloud disk). This brings us to the **"Gold Standard"** of Kubernetes storage: **PersistentVolumes (PV)** and **PersistentVolumeClaims (PVC)**. # **Persistent Volumes(PV) and Persistent Volumes Claim (PVC)** ## The What ? ### **Persistent Volumes (PV):** A **Persistent Volume (PV)** represents a piece of storage provisioned in the cluster often backed by **network or cloud storage** that continues to exist even if Pods are deleted or moved. You can think of a PV as **block storage or file storage attached to the Kubernetes cluster**, where the cluster has full control to allocate this storage to incoming requests. Since this storage lives **outside the Kubernetes cluster environment** (for example, cloud disks or network-attached storage), it is **not affected by Pod restarts, node failures, or most Kubernetes-level failures**. ### **Persistent Volumes Claim (PVC):** A **Persistent Volume Claim (PVC)** is a request for storage made by an application running inside the Kubernetes cluster. Instead of directly attaching a storage disk, a Pod asks for storage by specifying: * How much space it needs * The type of access required (read/write) * The storage characteristics it expects Kubernetes then looks for a suitable **Persistent Volume (PV)** that matches this request and binds it to the PVC. You can think of a PVC as an **interface between your application and the actual storage**, allowing Pods to use persistent storage without needing to know where or how it is physically provisioned. ### ❓ Why PVC request is needed ? PVCs introduce a **clean separation between applications and storage**. Different applications have very different storage needs: * Databases need **high I/O and low latency** * Logs or backups can work with **lower-performance storage** Instead of hard-coding storage details into applications, a PVC allows the app to simply say: “I need this much storage, with this access pattern.” Kubernetes then fulfills that request by binding the PVC to an appropriate PV. This approach keeps applications **portable, flexible, and storage-agnostic**, while allowing storage performance to be controlled through PVC requests. Here is a quick look at how they connect: * **Pod** points to → **PVC** * **PVC** binds to → **PV** * **PV** points to → **Actual Disk** Each layer has a clear responsibility, and none of them need to know the internal details of the others. ## 🔐 Access Modes Both **Persistent Volumes (PV)** and **Persistent Volume Claims (PVC)** define something called **access modes**. Access modes describe **how a volume can be accessed by Pods** for example, whether it can be mounted as read-only, read-write, or shared across multiple nodes. The available access modes are explained below * **ReadWriteOnce (RWO):** The volume can be mounted as read-write by a **single** node. (Like a USB drive plugged into one laptop). * **ReadOnlyMany (ROX):** The volume can be mounted read-only by **many** nodes. (Like a shared CD-ROM). * **ReadWriteMany (RWX):** The volume can be mounted as read-write by **many** nodes. (Like a shared network folder/Dropbox). When you provision **PV and PVC for a highly available application** that runs across multiple nodes, choosing the **right access mode** becomes critical. For example, If the volume is **read-only** and shared across Pods **across nodes**, you can use **ReadOnlyMany (ROX)**, If the volume needs to be **read and written** by multiple Pods **across nodes**, you must use **ReadWriteMany (RWX)** > What does “Many” mean here? > > “Many” refers to the **number of nodes**, not Pods. > > * **RWO** → Only **one node** can read/write to the volume at a time. i.e all the pods in the node can access this volume > > * **RWX** → **Multiple nodes** can read/write to the same volume simultaneously. i.e pods across nodes can access the volume > An important detail to note here is a **Persistent Volume (PV)** can be configured with **multiple access modes**, indicating what it **supports**. A **Persistent Volume Claim (PVC)**, on the other hand, requests **a specific access mode** based on the application’s requirement. If the requested access mode in the PVC is **supported by the PV**, Kubernetes approves the claim and binds them together. > ### 🧠 Key Takeaway > > **PV advertises what it can support, and PVC asks for what it needs.** Binding happens only when both agree on the access mode. ## Example Kubernetes Resource Definition File ### Persistent Volume (PV) ```yaml apiVersion: v1 kind: PersistentVolume metadata: name: app-pv # Name of the PV spec: capacity: storage: 10Gi # Size of the volume accessModes: - ReadWriteOnce # Supported access mode - ReadWriteMany persistentVolumeReclaimPolicy: Retain # Retain data even if PVC is deleted storageClassName: ebs-sc # Must match PVC csi: driver: ebs.csi.aws.com # AWS EBS CSI driver volumeHandle: vol-0abcd1234efgh5678 # Actual EBS volume ID from AWS volumeMode: Filesystem # Mount as a filesystem ``` ### Persistent Volume (PV) ```yaml apiVersion: v1 kind: PersistentVolumeClaim metadata: name: app-pvc # Name of the PVC spec: accessModes: - ReadWriteOnce # Only one node can read/write resources: requests: storage: 10Gi # Amount of storage requested storageClassName: ebs-sc # Use this StorageClass ``` ### Consuming a PVC Inside a Pod ```yaml apiVersion: v1 kind: Pod metadata: name: app-pod # Name of the Pod spec: containers: - name: app-container image: nginx # Sample container image volumeMounts: - name: app-storage mountPath: /data # Path inside the container # The PVC will be mounted here volumes: - name: app-storage persistentVolumeClaim: claimName: app-pvc # Name of the PVC to use ``` # **StorageClasses:** Leveling Up to Dynamic Provisioning In the real world the admin don’t want to worry about creating 100’s of PV’s(Persistent Volumes) with different types and sizes. Here is where **StorageClasses** (SC) comes in ## How it works * The administrator creates a **StorageClass** in the cluster * The **StorageClass** is granted permission to create cloud provider disks (such as **AWS EBS** or **Azure Disk**) by installing the appropriate **CSI driver** in the cluster. * For example, in an **Amazon EKS** cluster, you install the **EBS CSI driver**, which enables Kubernetes to dynamically provision EBS volumes on demand whenever a PVC is created. * The PVC specifies **which StorageClass to use** * When the PVC is created, Kubernetes checks: * Is there an existing PV that matches this request? * If **no matching PV exists**, Kubernetes uses the StorageClass to **dynamically create a new PV**. * The newly created PV is **bound to the PVC**. * When a Pod references the PVC, the volume is **attached and mounted** into the Pod. For example: * A Pod requests **50Gi** * No existing PV matches * The StorageClass dynamically creates a new disk (EBS etc.) * The disk is attached to the Pod as a volume No manual PV creation needed 🎉 # Reclaim Policies Now comes an important question. Your application stores data in a PV via a PVC and that data can live forever. But what if: * You delete the application? * You no longer need the data? * You recreate the app with fresh data? Continuously creating new volumes is **not sustainable**, especially when storage costs add up. This is where **Reclaim Policies** come into play. Reclaim policies define **what happens to a Persistent Volume after the PVC is deleted** whether the data should be: * **Retained:** The PV is not deleted. It stays in "**Released”** state and will not be assigned to any new PVC request. An admin must manually clean up the data. This is the **safest** for production databases. * **Deleted:** The PV and the actual physical disk (AWS EBS, GCE PD) are deleted immediately. **Should be choosen with extra caution as the data will be wiped once PVC is deleted** * **Recycle(Deprecated):** It performs a basic `rm -rf /thevolume/*` and makes the PV available again. (Rarely used now). You control this with the `persistentVolumeReclaimPolicy` ## The Binding Process Kubernetes matches a PVC to a PV using these criteria: 1. **Capacity:** Does the PV have at least the amount requested? 2. **Access Mode:** Does the PV support the mode (RWO, RWX, etc.)? 3. **StorageClass:** Do they have the same StorageClass name? 4. **Selectors:** (Optional) Do the labels match? ## Example Kubernetes Resource Definition File ### StorageClass (Dynamic Provisioning) ```yaml apiVersion: storage.k8s.io/v1 kind: StorageClass metadata: name: ebs-sc # Name of the StorageClass provisioner: ebs.csi.aws.com # CSI driver that talks to AWS EBS parameters: type: gp3 # EBS volume type (gp2 / gp3 / io1, etc.) reclaimPolicy: Delete # What happens to the disk when PVC is deleted volumeBindingMode: WaitForFirstConsumer # Volume is created only when a Pod actually uses the PVC ``` # Common Scenarios ## The “Same Name” Scenario (ReclaimPolicy: Retain) Assume you have an application using a **Persistent Volume (PV)** named `pv-x`, and the reclaim policy is set to **Retain**. You deploy the app, it creates a **PVC**, and everything works fine. After some experimentation, you delete the application (which deletes the PVC). Later, you: * Create a **new application** with a **different PVC name**, but want to use the **same data** * Or simply **rename the PVC** for the same application So… what happens? When the PVC is deleted and the reclaim policy is **Retain**: * The **PV is NOT deleted.** The **underlying disk is preserved** * The PV moves into a **Released** state. The PV **still remembers the old PVC** it was bound to This is the important part 👇 A PV in the **Released** state **cannot be automatically reused** by a new PVC—even if: * The storage size matches * The access modes match * The StorageClass matches Kubernetes intentionally blocks this to **prevent accidental data leaks** between workloads. ### ❓ Why Can’t a New PVC Reuse the Same PV Automatically? Because the PV still contains Old application data and Metadata pointing to the previous PVC. Kubernetes assumes this data belongs to someone else. I won’t reattach it unless a human explicitly says so. That’s a **safety feature**, not a limitation. ### So How Can You Recover the Data? There are **two correct and safe ways** to recover data from a retained PV. **✅ Option 1: Manually Rebind the Existing PV (Most Common)** If you want the **same data** to be used by a new or renamed PVC: * **Manually edit the PV** and remove the old `claimRef` * Set the PV back to `Available` * Create a new PVC that matches: StorageSize, AccessMode, and Storage class * Once this is done, Kubernetes will bind the new PVC to the old PV—and **your data is recovered intact**. This is the **intended recovery flow** for `Retain`. **✅ Option 2: Create a New PV Pointing to the Same Disk** This is useful when you don’t want to touch the old PV object and you want more control over the new binding. * Identify the underlying disk (EBS volume, Azure Disk, etc.) * Create a **new PV definition** * Point it to the **same physical disk** * Create a new PVC that binds to this new PV The data remains untouched because the disk never changed—only the Kubernetes objects did. ## Do you know AWS EBS Does NOT Support RWX AWS **Elastic Block Storage (EBS)** does **not** support `ReadWriteMany (RWX)` access mode. why ? Because an EBS volume behaves like a **physical hard drive**. Once you attach a hard drive to **one machine**, you cannot plug that same drive into **multiple machines at the same time** and expect it to work safely. This is exactly how EBS works: * One EBS volume → one node * Read/write access → single node only That’s why EBS supports `ReadWriteOnce (RWO)`.This limitation is not a Kubernetes issue — it’s a **block storage limitation**. ### So What If You Need RWX Across Multiple Nodes? If your application needs to run on multiple nodes and read and write the same data simultaneously, then **block storage is the wrong tool**. This is where **file-based network storage** comes in. ### EFS: The RWX-Friendly Storage **Amazon Elastic File System (EFS)** is designed to solve exactly this problem. EFS behaves like a **shared network drive**. Multiple nodes can mount it and all nodes see the same file and Read/Write operations happens concurrently This wraps up our deep dive into **PV, PVC, and StorageClasses** — what they are, why they exist, and how Pods actually consume them. From here, you can start mapping these concepts to **real-world scenarios** and production use cases, not just lab setups.