The purpose of this document is to offer guidance on Red Hat Openshift Container Storage (OCS) in combination with Red Hat Openshift Container Platform (OCP) internal services for logging, metrics and registry. This document will show and explain in more detail how Openshift Container Storage offers relevant value for Openshift services.
Openshift Container Storage
Red Hat OCS is persistent software-defined storage integrated with and optimized for Red Hat OpenShift. It runs anywhere Red Hat OpenShift does: on-premise or in the public cloud. Built on Red Hat Ceph Storage, the Rook operator for Kubernetes storage orchestration and NooBaa multicloud object gateway technology, the platform offers tightly integrated, persistent data services for OpenShift and the hybrid multicloud. Dynamic, stateful, and highly available container-native storage can be provisioned and de-provisioned on demand as an integral part of the OpenShift administrator console.
OCP Services (internal stateful services supporting OCP itself)
- Openshift Logging (EFK Stack, Openshift’s log methodology)
- Openshift Metrics (Prometheus, Openshift’s monitoring tooling)
- Openshift Registry (Openshift’s built-in container image store)
Various OCP services maintain state, driving a need for persistent storage. Although each OCP service defaults to a basic way of storing data, this does not mean the default is also the most ideal way of storing the service’s stateful data. In most cases, the default offers basic functionality enabling the service to function. During failure scenarios, this can lead to negative consequences, in a worst case even an entire loss of an Openshift service’s state. To prevent that from happening, we recommend the use of Openshift Container Storage for OCP services in certain situations.
Table-1 below shows the defaults and our recommended storage options for each OCP service component.
Note, that when OCP is deployed in the AWS public cloud, the default IPI installed storage option for OCP Registry is sufficient and is our recommendation. For the other OCP services, as shown in table-1 below, other recommendations apply.
Table-1: Supported Infrastructures and service storage repository recommendations
OpenShift Container Platform Logging
OpenShift Logging is a key service of the OpenShift Container Platform monitoring stack. Data persistence for the logging service is strongly recommended. Data persistence is required to maintain the logs of all cluster components and processes.
OpenShift Container Platform Metrics
When running Openshift in production, it is advisable to persist the generated metrics. Metrics are collected for almost everything that happens inside of your Openshift installation. Persisting these Metrics enables you to retain this vital information no matter what happens inside of your cluster. By default, the Prometheus stack deployed to collect the metrics does not offer persistence.
OpenShift Container Platform Registry
The main purpose of the OCP registry is to serve and store container images. For this reason, it is a crucial component. A registry service requires back-end storage to store its contents. In AWS public cloud the default back-end storage is provided by using AWS S3. For all other infrastructure situations, a RWX PV is the Red Hat supported and recommended way to connect the OCP registry to persistent storage.
OpenShift Container Storage for Openshift Logging service summary
Whether you choose to deploy Openshift on-premise or in a public cloud, persistent storage is required to aggregate the logs from your OpenShift Container Platform cluster, including node system logs, application container logs, etc. The OCP cluster logging component logStore implements Elasticsearch to store and index all the logs for an OCP cluster.
Table-2 summarises the comparison between various OpenShift Logging Service storage options, including the current default storage option (emptyDir), OpenShift Container Storage, and AWS EBS.
Table-2: Storage options comparison for OCP Logging Service
OpenShift Logging is a key service of the OpenShift Container Platform monitoring stack. Data persistence for logging service is strongly recommended.
Summarized in table-3: OCS provides resilience across multiple availability zones, provides support for file, block, and object interfaces, and provides a common storage experience across OCP infrastructure types. These capabilities make OCS the preferred storage backend for the OCP logging service.
Table-3: Storage option comparison for OCP Logging Service across infrastructures
OCS for Openshift Metrics service summary
When running Openshift in production, it is advisable to persist the generated metrics. Metrics are collected for almost everything that happens inside of your Openshift installation. Whether you use these metrics for billing your users, doing capacity planning, or for monitoring purposes, persisting them enables you to retain this vital information no matter what happens inside of your cluster. By default, the Prometheus stack deployed to collect OCP metrics, is not configured for persistence.
No matter which persistence solution you choose, they are all configured in one central place: the cluster-monitoring-config ConfigMap in the openshift-monitoring project.
Using the ApacheBench software, we analyzed the performance impact of running Prometheus persistent on Openshift Container Storage (OCS) instead of running it ephemerally and found out that the impact is negligible. While there are other storage providers besides OCS that can provide storage to Prometheus, OCS is special as it provides a homogeneous interface on all OCP infrastructure types, removing many of the difficult choices you would need to make to keep your data safe and secure.
There are a variety of reasons to persist metrics data and protect from data loss. Fortunately, the performance impact of running the metrics stack on persistent storage has been measured as negligible. If you envision running OCP on multiple infrastructure types, using the default persistent storage class for that infrastructure carries the risk of learning and managing different storage functionality for each infrastructure platform. By choosing Openshift Container Storage you get a common storage experience providing uniform capabilities on-premises or on public clouds, that is quick to set up and easy to manage.
OCS for Openshift Registry service summary
The main purpose of an image-registry is to serve and store container images. A container image is a basic requirement for running containers within pods. For this reason, it is crucial to have access to a container image registry.
The registry service requires back-end storage to store its repository contents. This storage must be shared and available across all OCP cluster nodes. OCS can take care of this by design. In addition, data inside OCS is protected by internal replication to prevent data loss in the event of hardware failure.
OCP can run on many infrastructures, including public clouds, private clouds, and on-premises on VMware or bare metal clusters. The OCP Image Registry must be shared with all worker nodes in the cluster. Supported storage types are shared object storage (AWS) and shared RWX Persistent Volumes (PVs).
OpenShift Container Storage: openshift.com/storage
OpenShift | Storage YouTube Playlist
OpenShift Commons ‘All Things Data’ YouTube Playlist
To find out more about OpenShift Container Storage or to take a test drive, visit https://www.openshift.com/products/container-storage/.