Live migration of virtual machines (VMs) with OpenShift Virtualization on Red Hat OpenShift Service on AWS and Amazon FSx for NetApp ONTAP

• How to set FSxN as the default storage for a ROSA cluster
• Creating a VM that uses FSxN storage
• Migrating a VM to a new node

• AWS account
• Red Hat account
• IAM user with appropriate permissions to create and access ROSA cluster
• AWS CLI
• ROSA CLI
• OpenShift command-line interface (oc)
• Helm 3 documentation
• A HCP ROSA cluster (with at least 2 bare-metal worker nodes)
• OpenShift Virtualization installed on ROSA Cluster 
• Access to Red Hat OpenShift web console

This diagram shows the ROSA Hosted Control Plane (HCP) cluster deployed in multiple availability zones (AZs). ROSA clusters’ control plane nodes are in a service team virtual private cloud (VPC), while the compute/Worker nodes are in a VPC in the customer's account. Some of the worker nodes are metal instance type to support OpenShift Virtualization. The FSxN file system is deployed within the same VPC. NetApp Trident provisioner is installed in the ROSA cluster, allowing all the subnets of this VPC to connect to the file system. OpenShift Virtualization is installed using the operator from the OperatorHub. 

1. Ensure the default Storage Class is set to trident-csi. The following yaml was used to create the storage class.

    

   apiVersion: storage.k8s.io/v1
                                                   kind: StorageClass
                                                   metadata:
                                                     name: trident-csi
                                                   provisioner: csi.trident.netapp.io
                                                   parameters:
                                                     backendType: "ontap-nas"
                                                     fsType: "ext4"
                                                   allowVolumeExpansion: True
                                                   reclaimPolicy: Retain

    

   

    

   Note: Prior to creating the Storage class, the following yaml files were used to create a secret and the backend object using ontap-nas driver.

   apiVersion: v1
                                                   kind: Secret
                                                   metadata:
                                                     name: backend-fsx-ontap-nas-secret
                                                     namespace: trident
                                                   type: Opaque
                                                   stringData:
                                                     username: vsadmin
                                                     password: <SVM password configured while creating FSxN>
                                                   apiVersion: trident.netapp.io/v1
                                                   kind: TridentBackendConfig
                                                   metadata:
                                                     name: backend-fsx-ontap-nas
                                                     namespace: trident
                                                   spec:
                                                     version: 1
                                                     backendName: fsx-ontap
                                                     storageDriverName: ontap-nas
                                                     managementLIF: <Management DNS name>
                                                     dataLIF: <NFS DNS name>
                                                     svm: <SVM NAME>
                                                     credentials:
                                                       name: backend-fsx-ontap-nas-secret

2. Ensure the default VolumeSnapShotClasses is set as shown. The following yaml was used to create the VolumeSnapshotClass:

    

   kind: VolumeSnapshotClass
                                                           metadata:
                                                             name: fsx-snapclass
                                                           driver: csi.trident.netapp.io
                                                           deletionPolicy: Delete

    

   

    

    If the defaults are not set, you can set it up either from the console or from command line:

   $ oc patch storageclass trident-csi -p '{"metadata": {"annotations": {"storageclass.kubernetes.io/is-default-class": "true"}}}'
                                                           $ oc patch VolumeSnapshotClasses fsx-snapclass -p '{"metadata": {"annotations": {"snapshot.storage.kubernetes.io/is-default-class": "true"}}}'

1. From the ROSA console, create a VM. There are several templates available, but for the purposes of this scenario, we will use the Fedora VM.

   

    

   

    

2. Give the VM a name, and then click Customize Virtual Machine.

   

    

    

3. Select the Disks tab and click Add disks.

    

   

    

4. Change the name of the disk to whatever preferred title, and ensure that trident-csi is selected for storage class. Then click Save. 

   

    

5. After the options are saved, click Create VirtualMachine. 

   

Once the VM is created, you will want to review that all the objects necessary for it to run are present and accounted for. 

• Storage disks

  

• The file systems of the VM will show the partitions, file system type and mount points.

  

   

• 2 Persistent Volume Claims (PVCs) are created for the VM, one from the boot disk and one for the hot plug disk.

  

   

• The PVC for the boot disk shows that the Access mode is set to ReadWriteMany and the Storage Class is trident-csi.

  

   

• Similarly, the PVC for the hot-plug disk shows that the Access mode is ReadWriteMany, and the Storage Class is trident-csi.

  • In the screenshot below we can see that the pod for the VM has a Status of Running.

    

     

• Connect to the VM by clicking Open web console, and login using Guest Credentials.

  

   

  

   

• Issue the following commands:

  • $ df (to display information about the disk space usage on a file system).

  •  $ dd if=/dev/urandom of=random.dat bs=1M count=10240 (to create a file called random.dat in the home dir and fill it with random data).

• The disk is filled with 11 GB of data.

  

   

• Use vi to create a sample text file that we will use to test.

  

  
   

1. The VM comes back to the running state with the exact same filesystems, PVCs and files in the filesystems.

   

    

   

    

   

Live migration refers to the process of moving a running VM from one physical host to another host without disrupting normal operations, causing any downtime, or causing other adverse effects for the end user.

Live migration is considered a major step in virtualization. It allows an entire VM to be moved with a running operating system (OS), memory, storage, and network connectivity from their current node to the destination.

Below we will see how to perform a live migration of the VM from the current node to a new node.

1. Note the node on which the VM is running.

   

    

2. Click on the 3 dots in the far right of the VM entry and select Migrate.

   

    

3. On the Overview page, you can see that the migration has succeeded, and the Status has changed to Succeeded.

   

    

4. After the completion of the Live Migration, the VM is now on a different node.

   

    

5. Open the web console and view the contents of the disks. It still has the same 2 files that we previously created before live migration.

   

    

   

    

    

6. The storage for the VM on the new node still shows the same disks.

   

   Also, the PVCs are the same.

   

    

7. Volumes associated with the VM pod are also the same (2 PVCs) as before.

You have now successfully conducted a live migration of virtual machines using Amazon FSxN! 

• Troubleshoot with Red Hat support
• Troubleshoot with Amazon support