Red Hat blog
Starting in Red Hat OpenShift 4.9 release, we are extending the infrastructure provider integration experience to a new platform, Azure Stack Hub, from Microsoft’s Azure Stack portfolio of products.
This will allow Microsoft customers to deploy Red Hat OpenShift on a user-provisioned infrastructure in Azure Stack Hub. Red Hat is planning to support full-stack automated deployments in coming releases. As part of this new provider integration certification, Red Hat makes Azure Resource Manager templates available to the customer to simplify the process of creating the required infrastructure where to deploy OpenShift.
Azure Stack Hub extends Azure services and capabilities to your environment of choice, letting you run OpenShift in an on-premise environment and deliver Azure services and OpenShift capabilities in your data centre. For many customers, due to technological and regulatory obstacles, their workloads must remain on-premises. This integration will help customers to build their hybrid cloud strategy leveraging OpenShift to run their workloads on-premises, keeping the same processes and user experience as they were running these workloads in the Azure public cloud.
Deploying OpenShift on Azure Stack Hub using user-provisioned infrastructure
Detailed instructions on how to deploy OpenShift on Azure Stack Hub are available in docs.openshift.com as they are for other platforms. This post is doing a high-level overall overview of the steps required.
For this release, the user will need to provide the required infrastructure where OpenShift will be deployed. For customers who want to leverage infrastructure as code Azure provides a solution called Azure Resource Manager templates. As part of the instructions, we are providing opinionated templates as a reference architecture to easily create all the required infrastructure components for OpenShift, such as networking, storage, compute and others.
When using user-provisioned infrastructure the steps are the same as any other provider. An install-config.yaml needs to be created with the specific information for the Azure Stack Hub environment where we are deploying OpenShift. The documentation already provides a sample that the user will customize to match their needs.
Using the OpenShift installation program we will generate the Kubernetes manifest for the cluster:
$ openshift-install create manifests
INFO Credentials loaded from file "/home/mak/.azure/osServicePrincipal.json"
INFO Consuming Install Config from target directory
WARNING Making control-plane schedulable by setting MastersSchedulable to true for Scheduler cluster settings
INFO Manifests created in: manifests and openshift
As we are deploying the cluster using an infrastructure provisioned by the user, we need to do some additional changes to the generated manifests. We are going to remove the manifest files that define the control plane and worker machines as we are going to manually manage those.
The next step is to manually create the cloud credentials from the release image the installation program is built to use:
$ oc adm release extract quay.io/openshift-release-dev/ocp-release:4.9.0-rc.8-x86_64 --credentials-requests --cloud=azure --to=credentials-request
$ ll credentials-request/
total 20
-rw-rw-r-- 1 mak mak 568 Oct 18 21:39 0000_26_cloud-controller-manager-operator_14_credentialsrequest-azure.yaml
-rw-rw-r-- 1 mak mak 591 Oct 18 21:39 0000_30_machine-api-operator_00_credentials-request.yaml
-rw-rw-r-- 1 mak mak 656 Oct 18 21:39 0000_50_cluster-image-registry-operator_01-registry-credentials-request-azure.yaml
-rw-rw-r-- 1 mak mak 641 Oct 18 21:39 0000_50_cluster-ingress-operator_00-ingress-credentials-request.yaml
-rw-rw-r-- 1 mak mak 611 Oct 18 21:39 0000_50_cluster-storage-operator_03_credentials_request_azure.yaml
We need to generate now the corresponding Secrets using the secretRef information from each CredentialsRequest file, and after that, we can create the Ignition configuration files:
$ openshift-install create ignition-configs
INFO Consuming Worker Machines from target directory
INFO Consuming Common Manifests from target directory
INFO Consuming Master Machines from target directory
INFO Consuming Openshift Manifests from target directory
INFO Consuming OpenShift Install (Manifests) from target directory
INFO Ignition-Configs created in: . and auth
We are now ready to create all the required infrastructure using the provided Azure Resource Manager templates and bootstrap the cluster:
$ openshift-install wait-for bootstrap-complete --log-level debug
DEBUG OpenShift Installer 4.9.0-rc.8
DEBUG Built from commit 6e5b992ba719dd4ea2d0c2a8b08ecad45179e553
INFO Waiting up to 20m0s for the Kubernetes API at https://api.mak-blog.ppe.upi.azurestack.devcluster.openshift.com:6443...
INFO API v1.22.0-rc.0+894a78b up
INFO Waiting up to 30m0s for bootstrapping to complete...
DEBUG Bootstrap status: complete
INFO It is now safe to remove the bootstrap resources
DEBUG Time elapsed per stage:
DEBUG Bootstrap Complete: 10m42s
DEBUG API: 1s
INFO Time elapsed: 10m42s
At this point the OpenShift Cluster will be ready to use and worker nodes can be added to the cluster.
$ oc get clusteroperators
NAME VERSION AVAILABLE PROGRESSING DEGRADED SINCE MESSAGE
authentication 4.9.0-rc.8 True False False 29h
baremetal 4.9.0-rc.8 True False False 4d17h
cloud-controller-manager 4.9.0-rc.8 True False False 4d17h
cloud-credential 4.9.0-rc.8 True False False 4d17h
cluster-autoscaler 4.9.0-rc.8 True False False 4d17h
config-operator 4.9.0-rc.8 True False False 4d17h
console 4.9.0-rc.8 True False False 29h
csi-snapshot-controller 4.9.0-rc.8 True False False 4d17h
dns 4.9.0-rc.8 True False False 4d17h
etcd 4.9.0-rc.8 True False False 4d17h
image-registry 4.9.0-rc.8 True False False 4d17h
ingress 4.9.0-rc.8 True False False 4d16h
insights 4.9.0-rc.8 True False False 4d17h
kube-apiserver 4.9.0-rc.8 True False False 4d17h
kube-controller-manager 4.9.0-rc.8 True False False 4d17h
kube-scheduler 4.9.0-rc.8 True False False 4d17h
kube-storage-version-migrator 4.9.0-rc.8 True False False 4d17h
machine-api 4.9.0-rc.8 True False False 4d17h
machine-approver 4.9.0-rc.8 True False False 4d17h
machine-config 4.9.0-rc.8 True False False 4d17h
marketplace 4.9.0-rc.8 True False False 4d17h
monitoring 4.9.0-rc.8 True False False 4d16h
network 4.9.0-rc.8 True False False 4d17h
node-tuning 4.9.0-rc.8 True False False 4d17h
openshift-apiserver 4.9.0-rc.8 True False False 29h
openshift-controller-manager 4.9.0-rc.8 True False False 4d17h
openshift-samples 4.9.0-rc.8 True False False 4d17h
operator-lifecycle-manager 4.9.0-rc.8 True False False 4d17h
operator-lifecycle-manager-catalog 4.9.0-rc.8 True False False 4d17h
operator-lifecycle-manager-packageserver 4.9.0-rc.8 True False False 29h
service-ca 4.9.0-rc.8 True False False 4d17h
storage 4.9.0-rc.8 True False False 4d17h
We can log in to the OpenShift Console and check the status of the Cluster.
Check if nodes are ready to run workloads.
Or check that we can consume storage volumes directly from our Azure Stack Hub environment.
Conclusion
This new provider integration will enable existing or new Azure customers to deploy and run OpenShift clusters on their own autonomous cloud on-premises the same way they were doing on the public cloud.
Red Hat’s hybrid cloud experience will let customers use consistent tools and models while giving them the flexibility to choose between Azure and Azure Stack Hub when building and deploying their applications.
We are really excited to see how customers take advantage of this new feature and continue working with them on their hybrid cloud journey.