What is Composable OpenShift?

Composable OpenShift refers to a change in our philosophy and approach to evolving the Red Hat OpenShift product towards a more flexible installation model. The earliest versions of OpenShift 4 were delivered in what was effectively a one-size-fits-all package, which meant regardless of intended usage or deployment environment, every cluster ran the same components. This was followed by a few other packaging options, such as Single Node OpenShift (SNO) that were tailored towards specific resource footprints and use cases. For example, SNO is well suited for smaller footprints with limited resources and small, remote locations where edge computing can accelerate highly localized workloads. However, there wasn’t much control for the cluster administrators to tailor the set of cluster components to their precise needs, unless it happened to align with one of OpenShift’s product packages.

Composable OpenShift seeks to enable cluster administrators or independent software vendors (ISVs) to tailor the set of components to their specific needs and use cases. Of course, there are many core components which are required for an OpenShift cluster to function, but there are other components which are only needed when running in certain environments or for certain use cases.

To understand the constraints in place, it is useful to understand how OpenShift is installed and self-managed. At the heart of the cluster is a component called the Cluster Version Operator (CVO). The CVO is responsible for installing and managing a set of operators called Cluster Operators. These are the components you may be more familiar with, such as the Machine Config Operator, the OpenShift API Server Operator, the Cluster Bare Metal Operator, and so on. While you can configure what these operators do, such as using the Cluster Bare Metal Operator to manage machines, you could not previously control whether the operator itself was running on your cluster. This meant that even if you had no need for the capabilities of a component such as the Cluster Bare Metal Operator because you were not managing bare metal machines, you had no choice but to run that operator because it was included by default and was deployed by the CVO.

In our first phase of evolving towards a composable cluster experience, we’ve made it possible to configure the CVO to exclude certain components from the installation. This means rather than having idle cluster operators running in your cluster, you can prevent their deployment entirely.

What are the benefits?

There are two primary motivations for disabling an unneeded component in a cluster: resource cost and risk profile.

Cost is relatively obvious. The fewer components your cluster is running, the less infrastructure you need to dedicate to it. Every pod OpenShift doesn’t have to run represents an opportunity to run another workload resource, or scale down the size of the control plane nodes. Previously while it was possible to limit the amount of activity some of these optional capabilities performed when the functionality was not needed, there was no escaping the minimal cost of at least running the cluster operator itself. Now, for capabilities enabled as optional, there is no overhead associated with unused capabilities.

Reducing the cluster’s risk profile is a more subtle benefit – specifically, the fewer components a cluster runs, the less likely it is to be impacted by any given bug. In practice, this may mean a particular Common Vulnerabilities and Exposures (CVE) has no chance of being exploited on a given cluster since the component is not running. This turns a “high pressure urgent upgrade” situation to “no impact” just by confirming the cluster never enabled the capability in question.

Another way cluster administrators benefit from this reduced risk profile is in the form of conditional upgrade edges. Conditional upgrade edges describe a situation whereby an upgrade to a new version is not recommended to clusters with particular characteristics. The upgrade is generally safe for most clusters, but for clusters with certain configurations or other unique situations, it may not be recommended. This is another scenario whereby the fewer components a cluster runs, the less likely it is the cluster will be impacted by those conditions. For example, if there is a bug that impacts the Console, clusters that don’t run the Console could still be recommended to upgrade to a new version, whereas other clusters might be recommended to wait for a newer version before upgrading.

What can Composable OpenShift Do Today?

So far, we have introduced seven capabilities which represent cluster operators and their related functionality which can be disabled at installation time:

• cluster baremetal operator (CBO)
• cluster CSI snapshot controller operator
• cluster storage operator
• console operator
• Insights operator
• marketplace operator
• samples operator

You can read more about each of these capabilities and the implications of disabling them here: https://docs.openshift.com/container-platform/4.12/installing/cluster-capabilities.html

Capabilities can either be enabled or disabled as a set, or individually. By default, OpenShift continues to install all components including the ones supporting these optional capabilities.  In addition, on upgrade, the cluster will pick up any new optional capabilities being introduced. However if you opt into one of the non-default capability sets, you can explicitly choose which capabilities to install. The primary capability set options are as follows:

• vCurrent: The recommended set of capabilities associated with the current version of OpenShift that the cluster is running. This automatically evolves as your cluster is upgraded to include new recommended capabilities.
• v4.12/v4.11/etc: This is a static set of capabilities that represents the recommended capabilities for a given OpenShift version. As the cluster is upgraded, the set of enabled capabilities will not evolve beyond this fixed set.
• None: All optional capabilities are disabled.

After picking the baseline capability set, additional individual capabilities can be enabled separately, extending the set of capabilities provided by the baseline set.

The initial capability set is defined in the install-config.yaml used during cluster installation. After installation, additional capabilities can be enabled by modifying the ClusterVersion resource. At all times the ClusterVersion status section reports which capabilities are currently enabled on the cluster based on the configured capability set and additional enabled capabilities.

apiVersion: config.openshift.io/v1
kind: ClusterVersion
metadata:
  name: version
spec:
  channel: stable-4.12
  clusterID: 8d92b82a-0ee8-4fb5-bc3f-c99bb5d5625f
  baselineCapabilitySet: vCurrent
status:
  capabilities:
        enabledCapabilities:
        - CSISnapshot
        - Console
        - Insights
        - Storage
        - baremetal
        - marketplace
        - openshift-samples
        knownCapabilities:
        - CSISnapshot
        - Console
        - Insights
        - Storage
        - baremetal
        - marketplace
        - openshift-samples

Figure 1. Example capabilities configuration and status portions of the ClusterVersion resource

An important consideration to keep in mind about optional capabilities is that they can only be disabled during the initial deployment, not after the cluster is running. Though you can later choose to enable a capability that you initially disabled during installation, once a capability is enabled, it cannot be disabled. This is a precaution imposed by the complexity of assuring there are no remaining side effects or dependencies broken by disabling a capability that was once enabled on a cluster. This means it is important to carefully consider whether a given capability should be enabled on the cluster or not.

Our expectation is that cluster providers/provisioners will leverage this new flexibility to provide customized clusters that are aligned to their user’s requirements. This can mean defining profiles like “headless clusters,” which disable the Console capability or “barebones” clusters which disable all optional capabilities to minimize resource cost for restricted use cases.

What is next for Composable OpenShift?

So far we have built a framework for defining and managing optional capabilities in OpenShift and defined an initial set of optional capabilities. In future releases, we anticipate expanding the set of capabilities which can be optionally disabled. Future candidates to make optional include the internal image registry and some of the other components that provide extended OpenShift API functionality beyond what is included in core Kubernetes.

Additional Information

• Streaming session introducing Composable OpenShift
• Enhancement Proposal describing the Optional Capability Architecture
• Product Documentation for Optional Capabilities