While Red Hat and Amazon Web Services (AWS) manage the ROSA service, the customer has certain responsibilities, including managing their data and their applications.
In a situation where the customer accidentally removes their application on OpenShift, there is no way to recover the application data unless it’s been backed up by the customer. In this post we will demonstrate how to back up an application project using Velero Backup. The OADP (OpenShift API for Data Protection) operator will be used to set up and install Velero on the OpenShift platform.
Steps to test the backup and recovery of a WordPress application running on ROSA:
- Deploy a ROSA cluster
- Install the OADP Operator
- Create credentials secret
- Install Velero using OADP
- Deploy a WordPress application
- Create a Backup
- Delete the application
- Restore from the backup
Install ROSA
Install the ROSA cluster by following the documentation.
Install the OADP Operator
Install the OADP Operator from OperatorHub. After selecting the defaults and clicking on Install, the OADP Operator will be installed in the oadp-operator project (namespace) as shown in the screenshot below.
Create credentials secret
Create a secret for the AWS credentials, which is used by Velero to access the Velero S3 bucket in your account. Use the “oc create secret” command as shown below to create the secret, where cloud-credentials.txt is the name of the file where the credentials are saved. Make sure that the cloud-credentials.txt file follows the AWS credentials template.
oc create secret generic cloud-credentials --namespace oadp-operator --from-file cloud=cloud-credentials.txt
Install Velero using OADP
Next we will create a Velero instance as follows from the CLI
First, create a file called velero.yaml with your Velero configuration. A sample velero.yaml file is included below:
apiVersion: konveyor.openshift.io/v1alpha1
kind: Velero
metadata:
name: example-velero
namespace: oadp-operator
spec:
olm_managed: true
backup_storage_locations:
- config:
profile: "default"
region: us-west-2
credentials_secret_ref:
name: cloud-credentials
namespace: oadp-operator
name: default
object_storage:
bucket: managed-velero-backups-1dc95bf4-3830-4231-904a-7782bd14f79e
prefix: "velero"
provider: aws
default_velero_plugins:
- aws
- openshift
enable_restic: true
volume_snapshot_locations:
- config:
profile: default
region: us-west-2
name: default
provider: aws
Then create that file by running the following command:
oc apply -f velero.yaml
Check to see if the Velero pods are up and running.
oc get pods -n oadp-operator
The output will look similar to this:
NAME READY STATUS RESTARTS AGE
oadp-default-aws-registry-774b459f89-c7977 1/1 Running 0 19s
oadp-operator-57985ccd67-mndhb 1/1 Running 0 22h
restic-cws5x 1/1 Running 0 21s
restic-dpj29 1/1 Running 0 21s
restic-xw8n2 1/1 Running 0 21s
velero-655b4bbf95-jssmk 1/1 Running 0 22s
Deploy a WordPress application
Deploy the WordPress application from https://kubernetes.io/docs/tutorials/stateful-application/mysql-wordpress-persistent-volume/
If the wordpress-mysql pod does not start, apply the fix mentioned here.
Update application files mysql-deployment.yaml and wordpress-deployment.yaml to use the gp2 StorageClass
mysql-deployment.yaml:
apiVersion: v1
kind: Service
metadata:
name: wordpress-mysql
labels:
app: wordpress
spec:
ports:
- port: 3306
selector:
app: wordpress
tier: mysql
clusterIP: None
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: mysql-pv-claim
labels:
app: wordpress
spec:
accessModes:
- ReadWriteOnce
storageClassName: gp2
resources:
requests:
storage: 20Gi
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: wordpress-mysql
labels:
app: wordpress
spec:
selector:
matchLabels:
app: wordpress
tier: mysql
strategy:
type: Recreate
template:
metadata:
labels:
app: wordpress
tier: mysql
spec:
containers:
- image: mysql:5.6
name: mysql
env:
- name: MYSQL_ROOT_PASSWORD
valueFrom:
secretKeyRef:
name: mysql-pass
key: password
ports:
- containerPort: 3306
name: mysql
volumeMounts:
- name: mysql-persistent-storage
mountPath: /var/lib/mysql
volumes:
- name: mysql-persistent-storage
persistentVolumeClaim:
claimName: mysql-pv-claim
wordpress-deployment.yaml:
apiVersion: v1
kind: Service
metadata:
name: wordpress
labels:
app: wordpress
spec:
ports:
- port: 80
selector:
app: wordpress
tier: frontend
type: LoadBalancer
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: wp-pv-claim
labels:
app: wordpress
spec:
accessModes:
- ReadWriteOnce
storageClassName: gp2
resources:
requests:
storage: 20Gi
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: wordpress
labels:
app: wordpress
spec:
selector:
matchLabels:
app: wordpress
tier: frontend
strategy:
type: Recreate
template:
metadata:
labels:
app: wordpress
tier: frontend
spec:
containers:
- image: wordpress:4.8-apache
name: wordpress
env:
- name: WORDPRESS_DB_HOST
value: wordpress-mysql
- name: WORDPRESS_DB_PASSWORD
valueFrom:
secretKeyRef:
name: mysql-pass
key: password
ports:
- containerPort: 80
name: wordpress
volumeMounts:
- name: wordpress-persistent-storage
mountPath: /var/www/html
volumes:
- name: wordpress-persistent-storage
persistentVolumeClaim:
claimName: wp-pv-claim
kustomization.yaml:
secretGenerator:
- name: mysql-pass
literals:
- password=test123
resources:
- mysql-deployment.yaml
- wordpress-deployment.yaml
Install the application using “oc apply -k ./” into the wordpress project, and expose the route for the service using “oc expose svc/wordpress”.
oc get pods,pvc,svc,route -n wordpress
NAME READY STATUS RESTARTS AGE
pod/wordpress-6c77879668-vxkxt 1/1 Running 0 11h
pod/wordpress-mysql-8599ff598-psbsf 1/1 Running 0 11h
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
persistentvolumeclaim/mysql-pv-claim Bound pvc-0ce99737-ae50-4784-a2ee-7ea0d2776d9b 20Gi RWO gp2 11h
persistentvolumeclaim/wp-pv-claim Bound pvc-6241249b-2eab-4051-8bdb-948253c8c1fd 20Gi RWO gp2 11h
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/wordpress LoadBalancer 172.30.183.22 ad67d6c27c3434d41805205b38f2ecc4-1422739893.us-west-2.elb.amazonaws.com 80:30599/TCP 22h
service/wordpress-mysql ClusterIP None <none> 3306/TCP 22h
NAME HOST/PORT PATH SERVICES PORT TERMINATION WILDCARD
route.route.openshift.io/wordpress wordpress-wordpress.apps.mytest.wgms.p1.openshiftapps.com wordpress 80 None
Enter the URL wordpress-wordpress.apps.mytest.wgms.p1.openshiftapps.com in a web browser and configure WordPress. Then create your first blog entry for testing.
Create a Backup
The backup.yaml file below instructs Velero to backup the namespace wordpress where our application is running. The backup will be named backup-1.
apiVersion: velero.io/v1
kind: Backup
metadata:
namespace: oadp-operator
name: backup-1
spec:
includedNamespaces:
- wordpress
From within the wordpress namespace run the following command:
oc apply -f backup.yaml
To determine when the backup has completed, check the Status Phase to see if it is marked as Completed
oc get backup -n oadp-operator
NAME AGE
backup-1 12h
Check the status of the backup as follows
oc describe backup -n oadp-operator
…
…
…
Status:
Completion Timestamp: 2021-09-01T18:26:10Z
Expiration: 2021-10-01T18:25:26Z
Format Version: 1.1.0
Phase: Completed
Progress:
Items Backed Up: 82
Total Items: 82
Start Timestamp: 2021-09-01T18:25:26Z
Version: 1
Events: <none>
Delete the application
To test that the backup was indeed successful, we will simulate a failure by deleting the wordpress project
oc delete project wordpress
project.project.openshift.io "wordpress" deleted
Quick check to see if the project has indeed been deleted
oc get pods,pvc -n wordpress
No resources found in wordpress namespace.
Restore from the backup
Let restore the backup using Velero as follows using the restore.yaml shown below. Here we are calling our restore restore-1, and under the spec we specify the name of the backup that we want to use ie. backup-1. We specifically call out the from backup-1 restore the namespace wordpress.
apiVersion: velero.io/v1
kind: Restore
metadata:
namespace: oadp-operator
name: restore-1
spec:
backupName: backup-1
includeNamespaces:
- wordpress
To restore the project run the following command:
oc apply -f restore.yaml
restore.velero.io/restore-1 created
Check if the restore can be seen under the oadp-operator namespace.
oc get restore -n oadp-operator
NAME AGE
restore-1 33s
Check that restore status has indeed changed to Completed using the oc describe restore restore-5 -n oadp-operator command.
oc describe restore restore-1 -n oadp-operator
…
…
…
Status:
Completion Timestamp: 2021-09-02T17:14:47Z
Phase: Completed
Progress:
Items Restored: 52
Total Items: 52
Start Timestamp: 2021-09-02T17:14:05Z
Warnings: 19
Events: <none>
Check if the pods, pvc, service,route names are seen under the wordpress namespace.
oc get pods,pvc,svc -n wordpress
NAME READY STATUS RESTARTS AGE
pod/wordpress-6c77879668-vxkxt 1/1 Running 0 15m
pod/wordpress-mysql-8599ff598-psbsf 1/1 Running 0 15m
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
persistentvolumeclaim/mysql-pv-claim Bound pvc-8e6a38ba-79cd-46fa-ac23-be43fc32c751 20Gi RWO gp2 15m
persistentvolumeclaim/wp-pv-claim Bound pvc-ae827f20-92b1-4ca5-84e1-657d3cf61166 20Gi RWO gp2 15m
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/wordpress LoadBalancer 172.30.146.20 a1e4531ad6fd44fefa57469c96db579d-1493697719.us-west-2.elb.amazonaws.com 80:30711/TCP 15m
service/wordpress-mysql ClusterIP None <none> 3306/TCP 15m
NAME HOST/PORT PATH SERVICES PORT TERMINATION WILDCARD
route.route.openshift.io/wordpress wordpress-wordpress.apps.mytest.wgms.p1.openshiftapps.com wordpress 80 None
We can now access the WordPress application using the URL wordpress-wordpress.apps.mytest.wgms.p1.openshiftapps.com As we can see below that the blog post that we had before we deleted the project has been restored.
Scheduling Backups:
Now that we have seen how to backup and restore our application using Velero, we will now look into scheduling the backup using the Schedule API available in the OADP Operator.
As shown below we have created a schedule called test-schedule. For this post we have scheduled a backup at 5 minutes (*/5 * * * *). Under the Template enter the VolumeSnapshotLocations as “default”, IncludedNamespaces as “wordpress”, and StorageLocation as “default”
This will schedule the backup from the OADP Operator as shown below.
Under the Backup tab of the OADP Operator we can see that the backups are being taken at the scheduled time.
Conclusion
In this post we saw how to backup and restore an application running in a ROSA cluster. We also looked into scheduling backups of our application namespace using the OADP Operator.
Please look into the OADP Operator github location for any new features being added to it.
Categories
