Enabling the AWS EFS CSI Driver Operator on ROSA

The Amazon Web Services Elastic File System (AWS EFS) is a Network File System (NFS) that can be provisioned on Red Hat OpenShift Service on AWS clusters. With the release of OpenShift 4.10 the EFS CSI Driver is now GA and available.

This is a guide to quickly enable the EFS Operator on ROSA to a Red Hat OpenShift on AWS (ROSA) cluster with STS enabled.

Note: The official supported installation instructions for the EFS CSI Driver on ROSA are available here .

Dynamic vs Static provisioning

The CSI driver supports both Static and Dynamic provisioning. Dynamic provisioning should not be confused with the ability of the Operator to create EFS volumes.

Dynamic provisioning

Dynamic provisioning provisions new PVs as subdirectories of a pre-existing EFS volume. The PVs are independent of each other. However, they all share the same EFS volume. When the volume is deleted, all PVs provisioned out of it are deleted too. The EFS CSI driver creates an AWS Access Point for each such subdirectory. Due to AWS AccessPoint limits, you can only dynamically provision 120 PVs from a single StorageClass/EFS volume.

Static provisioning

Static provisioning mounts the entire volume to a pod.

Prerequisites

• A Red Hat OpenShift on AWS (ROSA) 4.10 cluster
• The OC CLI
• The AWS CLI
• jq command
• watch command

Set up environment

1. export some environment variables

Prepare AWS Account

In order to use the AWS EFS CSI Driver we need to create IAM roles and policies that can be attached to the Operator.

1. Create an IAM Policy

2. Create the Policy

   This creates a named policy for the cluster, you could use a generic policy for multiple clusters to keep things simpler.

3. Create a Trust Policy

4. Create Role for the EFS CSI Driver Operator

5. Attach the Policies to the Role

Deploy and test the AWS EFS Operator

1. Create a Secret to tell the AWS EFS Operator which IAM role to request.

2. Install the EFS Operator

3. Wait until the Operator is running

4. Install the AWS EFS CSI Driver

5. Wait until the CSI driver is running

Prepare an AWS EFS Volume for dynamic provisioning

1. Run this set of commands to update the VPC to allow EFS access

2. Assuming the CIDR and SG are correct, update the security group

At this point you can create either a single Zone EFS filesystem, or a Region wide EFS filesystem

Creating a region-wide EFS

1. Create a region-wide EFS File System

2. Configure a region-wide Mount Target for EFS (this will create a mount point in each subnet of your VPC by default)

Creating a single-zone EFS

Note: If you followed the instructions above to create a region wide EFS mount, skip the following steps and proceed to “Create a Storage Class for the EFS volume”

1. Select the first subnet that you will make your EFS mount in (this will by default select the same Subnet your first node is in)

2. Create your zonal EFS filesystem

3. Create your EFS mount point

Create a Storage Class for the EFS volume and verify a pod can access it.

1. Create a Storage Class for the EFS volume

2. Create a namespace

3. Create a PVC

4. Create a Pod to write to the EFS Volume

   It may take a few minutes for the pod to be ready. If you see errors such as Output: Failed to resolve "fs-XXXX.efs.us-east-2.amazonaws.com" it likely means its still setting up the EFS volume, just wait longer.

5. Wait for the Pod to be ready

6. Create a Pod to read from the EFS Volume

7. Verify the second POD can read the EFS Volume

   You should see a stream of “hello efs”

Cleanup

1. Delete the Pods

2. Delete the Volume

3. Delete the Namespace

4. Delete the storage class

5. Delete the EFS Shared Volume via AWS

   Note: if you receive the error An error occurred (FileSystemInUse) wait a few minutes and try again.

   Note: if you created additional mount points for a regional EFS filesystem, remember to delete all of them before removing the file system

6. Detach the Policies to the Role

7. Delete the Role

8. Delete the Policy