Adding infrastructure nodes to an ARO cluster

This document shows how to set up infrastructure nodes in an ARO cluster and move infrastructure related workloads to them. This can help with larger clusters that have resource contention between user workloads and infrastructure workloads such as Prometheus.

Microsoft reference: https://learn.microsoft.com/en-us/azure/openshift/howto-infrastructure-nodes

You can find the original (and more detailed) document describing the process for a self-managed OpenShift Container Platform cluster here

Prerequisites

• Azure Red Hat OpenShift cluster
• Helm CLI

Create Infra Nodes

We’ll use the MOBB Helm chart to add ARO machinesets for infra nodes. It looks up an existing machineset to collect cluster-specific settings, then creates a new machineset for infra nodes with matching provider settings.

1. Add the MOBB chart repository to your Helm

2. Update your repositories

3. Install the mobb/aro-machinesets Chart with parameters to create infra nodes

   Create a values.yaml file like this:

4. Wait for the new nodes to be available

Moving Infra workloads

Ingress

1. Set the nodePlacement on the ingresscontroller to node-role.kubernetes.io/infra and increase the replicas to match the number of infra nodes

2. Check the Ingress Controller Operator is starting pods on the new infra nodes

Registry

1. Set the nodePlacement on the registry to node-role.kubernetes.io/infra

2. Check the Registry Operator is starting pods on the new infra nodes

Cluster Monitoring

1. Configure the cluster monitoring stack to use the infra nodes

   Note: This will override any other customizations to the cluster monitoring stack, so you may want to merge your existing customizations into this before running the command.

2. Check the OpenShift Monitoring Operator is starting pods on the new infra nodes

   some Pods like prometheus-operator will remain on master nodes.

   sample output