Provisioning baremetal OpenShift clusters using RHACM with GitOps leveraging on-premise Assisted Installer

Recently, I published the blog Provisioning OpenShift clusters using GitOps with ACM, explaining how to create OpenShift clusters with RHACM using GitOps with ArgoCD. The OpenShift installation type was IPI and valid for most platforms: Azure, AWS, GCP, vSphere, etc., but not for baremetal. If you've ever installed an OpenShift cluster in baremetal and disconnected, you know how different it is from any other installation.

In this blog, I'll explain how to deploy a baremetal OpenShift cluster with Assisted Installer using RHACM and GitOps with ArgoCD. If you are not familiar with deploying OpenShift clusters with RHACM and GitOps, I recommend reading the article I wrote: GitOps for organizations: provisioning and configuring OpenShift clusters automatically. I also highly recommend reading the blog Managing OCP Infrastructures Using GitOps, which I used the first time I deployed this solution.

Solution overview

I'll use OpenShift GitOps and RHACM in the same way as I did in Provisioning OpenShift clusters using GitOps with ACM. I'll upload the Kubernetes objects to a git repository, ArgoCD will synchronize these objects to our OpenShift cluster, and RHACM will deploy the cluster leveraging Bare Metal Operator, Ironic, and Assisted Installer.

I'll create the BareMetalHosts (BMH), which are Metal³ Custom Resource Definitions (CRDs) that define a physical host and its properties. The BMHs will connect to the baseboard management controller (BMC) physical nodes using the Redfish protocol. The node's network will be statically configured using NMstateConfig. The OpenShift cluster will be deployed with Assisted Installer using the BareMetalHosts created.

Don't start creating all the objects. As many resources are involved, I recommend creating them one at a time and checking their status. Start by checking the prerequisites in the RHACM documentation before creating an infrastructure environment, and enable the Central Infrastructure Management service, which is provided with the Multicluster Engine.

Once you satisfy the prerequisites, move to the RHACM console. In the Infrastructure Environment, create and connect the Baremetal Hosts to the host's BMC with Redfish. Next, deploy a cluster using the Baremetal Hosts (existing discovered hosts) following Creating your cluster in ACM with the console. Check the objects created in the deployment, move to the command line, and deploy another cluster creating the same objects with other parameters following Creating your cluster in ACM with the command line. After that, upload the YAML files to your Git repo and create an ArgoCD Application to sync the objects.

The last step is to create a Helm chart with all the objects as templates and an ApplicationSet to create an Application per cluster, as seen in Provisioning OpenShift clusters using GitOps with ACM.

Baremetal Operator, Ironic, and Assisted Installer in RHACM

• Bare Metal Operator is the main component that interfaces with the Ironic API for all operations needed to provision baremetal hosts, such as hardware capabilities inspection, operating system installation, and re-initialization when restoring a baremetal machine to its original status.
• Ironic is a service for automating the provisioning and lifecycle management of bare metal machines.
• The OpenShift Assisted Installer provides for easy provisioning of new bare metal machines and creating OpenShift clusters. The Assisted Installer checks that all the hosts meet the requirements and triggers the OpenShift Container Platform cluster deployment. All the nodes (BMHs) have the Red Hat Enterprise Linux CoreOS (RHCOS) image written to the disk.
• The Assisted Image Service customizes and serves RHCOS images for the Assisted Installer Service. It downloads a set of RHCOS images on startup based on config.

The assisted installer provisioning workflow:

Objects

In the RHACM documentation chapter Creating your cluster with the command line, you can get all the objects needed to deploy the cluster with Assisted Installer:

• Namespace
• ClusterImageSet
• ClusterDeployment
• AgentClusterInstall
• NMStateConfig
• BaremetalHost
• InfraEnv
• KlusterletAddonConfig
• ManagedCluster

Useful tips

• As noted above, check the prerequisites, especially the connectivity between out-of-band management host IP addresses and RHACM. For virtualmedia, you'll need to open the port 6183. Troubleshooting with virtualmedia, assisted-agent, and ignition can be difficult, so try to get access to the host ILO and a virtual terminal. This will make it much easier to troubleshoot.
• Don't create all the objects at the same time. Create them one at a time and check their status.
• Contact Red Hat Professional services for assistance. Although we openly share knowledge by publishing blogs, solutions, and articles like this one, each environment has its own customizations and challenges, and we have wide experience with many customers worldwide.

Wrap up

The above steps deployed a baremetal OpenShift cluster using RHACM and an on-premises assisted installer.

First, check the prerequisites. Once the prerequisites are in place, move to the RHACM console, create the baremetal hosts, and check their status. After that, deploy an OpenShift cluster using Assisted Installer and the baremetal hosts. If the deployment is successful, move to the command line and create the objects one at a time, checking their status. Finally, create the objects in a Git repository and synchronize them to the RHACM cluster using ArgoCD.

If you want to automate baremetal OpenShift cluster deployments as a self-service resource, you can create a Helm Chart with all the objects and an ArgoCD ApplicationSet to deploy each cluster using the Helm chart as demonstrated in Provisioning OpenShift clusters using GitOps with ACM.