> Note: This example application deployment assumes you already have an OpenShift test cluster or have followed the instructions in the Deploying OpenShift Container Storage 4 to OpenShift 4 Blog to set up an OpenShift Container Platform 4.2.14+ cluster using OpenShift Container Storage 4.
In this Blog the ocs-storagecluster-ceph-rbd
storage class will be used by an OCP application + database deployment to create RWO (ReadWriteOnce) persistent storage. The persistent storage will be a Ceph RBD (RADOS Block Device) volume (object) in the Ceph pool ocs-storagecluster-cephblockpool
.
To do so we have created a template file, based on the OpenShift rails-pgsql-persistent
template, that includes an extra parameter STORAGE_CLASS
that enables the end user to specify the storage class the Persistent Volume Claim (PVC) should use. Feel free to download https://raw.githubusercontent.com/red-hat-storage/ocs-training/master/o… to check on the format of this template. Search for STORAGE_CLASS
in the downloaded content.
oc new-project my-database-app
curl <a href="https://raw.githubusercontent.com/red-hat-storage/ocs-training/master/ocp4ocs4/configurable-rails-app.yaml">https://raw.githubusercontent.com/red-hat-storage/ocs-training/master/ocp4ocs4/configurable-rails-app.yaml</a> | oc new-app -p STORAGE_CLASS=ocs-storagecluster-ceph-rbd -p VOLUME_CAPACITY=5Gi -f -
After the deployment is started you can monitor with these commands.
oc status
oc get pvc -n my-database-app
This step could take 5 or more minutes. Wait until there are 2 Pods in Running
STATUS and 4 Pods in Completed
STATUS as shown below.
watch oc get pods -n my-database-app
Example output:
NAME READY STATUS RESTARTS AGE
postgresql-1-deploy 0/1 Completed 0 5m48s
postgresql-1-lf7qt 1/1 Running 0 5m40s
rails-pgsql-persistent-1-build 0/1 Completed 0 5m49s
rails-pgsql-persistent-1-deploy 0/1 Completed 0 3m36s
rails-pgsql-persistent-1-hook-pre 0/1 Completed 0 3m28s
rails-pgsql-persistent-1-pjh6q 1/1 Running 0 3m14s
You can exit by pressing Ctrl+C
Once the deployment is complete you can now test the application and the persistent storage on Ceph. Your HOST/PORT
will be different.
oc get route -n my-database-app
Example output:
NAME HOST/PORT PATH SERVICES PORT TERMINATION WILDCARD
rails-pgsql-persistent rails-pgsql-persistent-my-database-app.apps.cluster-a26e.sandbox449.opentlc.com rails-pgsql-persistent
Copy your rails-pgsql-persistent
route (different than above) to a browser window to create articles. You will need to append /articles
to the end.
Example http:///articles
Enter the username
and password
below to create articles and comments. The articles and comments are saved in a PostgreSQL database which stores its table spaces on the Ceph RBD volume provisioned using the ocs-storagecluster-ceph-rbd
storageclass during the application deployment.
username: openshift
password: secret
Lets now take another look at the Ceph ocs-storagecluster-cephblockpool
created by the ocs-storagecluster-ceph-rbd
Storage Class. Log into the toolbox pod again.
TOOLS_POD=$(oc get pods -n openshift-storage -l app=rook-ceph-tools -o name)
oc rsh -n openshift-storage $TOOLS_POD
Run the same Ceph commands as before the application deployment and compare to results in prior section. Notice the number of objects in ocs-storagecluster-cephblockpool
has increased. The third command lists RBDs and we should now have two RBDs.
ceph df
rados df
rbd -p ocs-storagecluster-cephblockpool ls | grep vol
You can exit the toolbox by either pressing Ctrl+D
or by executing exit
.
Matching PVs to RBDs
A handy way to match persistent volumes to Ceph RBDs is to execute:
oc get pv -o 'custom-columns=NAME:.spec.claimRef.name,PVNAME:.metadata.name,STORAGECLASS:.spec.storageClassName,VOLUMEHANDLE:.spec.csi.volumeHandle'
Example output:
NAME PVNAME STORAGECLASS VOLUMEHANDLE
ocs-deviceset-0-0-gzxjb pvc-1cf104d2-2033-11ea-ac56-0a9ccb4b29e2 gp2
ocs-deviceset-1-0-s87xm pvc-1cf33c42-2033-11ea-ac56-0a9ccb4b29e2 gp2
ocs-deviceset-2-0-zcjk4 pvc-1cf4f825-2033-11ea-ac56-0a9ccb4b29e2 gp2
db-noobaa-core-0 pvc-3008e684-2033-11ea-a83b-065b3ec3da7c ocs-storagecluster-ceph-rbd 0001-0011-openshift-storage-0000000000000001-3c0bb177-2033-11ea-9396-0a580a800406
postgresql pvc-4ca89d3d-2060-11ea-9a42-02dfa51cba90 ocs-storagecluster-ceph-rbd 0001-0011-openshift-storage-0000000000000001-4cbba393-2060-11ea-9396-0a580a800406
rook-ceph-mon-a pvc-cac661b6-2032-11ea-ac56-0a9ccb4b29e2 gp2
rook-ceph-mon-b pvc-cde2d8b3-2032-11ea-ac56-0a9ccb4b29e2 gp2
rook-ceph-mon-c pvc-d0efbd9d-2032-11ea-ac56-0a9ccb4b29e2 gp2
lab-ossm-hub-data pvc-dc1d4bdc-2028-11ea-ad6c-0a9ccb4b29e2 gp2
The second half of the VOLUMEHANDLE
column mostly matches what your RBD is named inside of Ceph. All you have to do is append csi-vol-
to the front like this:
Get the full RBD name of our postgreSQL PV in one command
oc get pv pvc-4ca89d3d-2060-11ea-9a42-02dfa51cba90 -o jsonpath='{.spec.csi.volumeHandle}' | cut -d '-' -f 6- | awk '{print "csi-vol-"$1}'
> Note: You will need to adjust the above command to use your
PVNAME
name
Example output:
csi-vol-4cbba393-2060-11ea-9396-0a580a800406
Now we can check on the details of our RBD from inside of the tools pod:
TOOLS_POD=$(oc get pods -n openshift-storage -l app=rook-ceph-tools -o name)
oc rsh -n openshift-storage $TOOLS_POD rbd -p ocs-storagecluster-cephblockpool info csi-vol-4cbba393-2060-11ea-9396-0a580a800406
Example output:
rbd image 'csi-vol-4cbba393-2060-11ea-9396-0a580a800406':
size 5 GiB in 1280 objects
order 22 (4 MiB objects)
snapshot_count: 0
id: 95e4f3973e8
block_name_prefix: rbd_data.95e4f3973e8
format: 2
features: layering
op_features:
flags:
create_timestamp: Tue Dec 17 00:00:57 2019
access_timestamp: Tue Dec 17 00:00:57 2019
modify_timestamp: Tue Dec 17 00:00:57 2019
> Note: You will need to adjust the above command to use your
RBD
name
Resources and Feedback
To find out more about OpenShift Container Storage or to take a test drive, visit https://www.openshift.com/products/container-storage/.
If you would like to learn more about what the OpenShift Container Storage team is up to or provide feedback on any of the new 4.2 features, take this brief 3-minute survey.
About the author
More like this
Browse by channel
Automation
The latest on IT automation that spans tech, teams, and environments
Artificial intelligence
Explore the platforms and partners building a faster path for AI
Open hybrid cloud
Explore how we build a more flexible future with hybrid cloud
Security
Explore how we reduce risks across environments and technologies
Edge computing
Updates on the solutions that simplify infrastructure at the edge
Infrastructure
Stay up to date on the world’s leading enterprise Linux platform
Applications
The latest on our solutions to the toughest application challenges
Original shows
Entertaining stories from the makers and leaders in enterprise tech
Products
- Red Hat Enterprise Linux
- Red Hat OpenShift
- Red Hat Ansible Automation Platform
- Cloud services
- See all products
Tools
- Training and certification
- My account
- Developer resources
- Customer support
- Red Hat value calculator
- Red Hat Ecosystem Catalog
- Find a partner
Try, buy, & sell
Communicate
About Red Hat
We’re the world’s leading provider of enterprise open source solutions—including Linux, cloud, container, and Kubernetes. We deliver hardened solutions that make it easier for enterprises to work across platforms and environments, from the core datacenter to the network edge.
Select a language
Red Hat legal and privacy links
- About Red Hat
- Jobs
- Events
- Locations
- Contact Red Hat
- Red Hat Blog
- Diversity, equity, and inclusion
- Cool Stuff Store
- Red Hat Summit