Deploying OpenShift 4.4 to VMware vSphere 7

Many organizations have a “virtual first” policy when deploying applications on premises, and, as you would expect, VMware’s vSphere hypervisor, being the most prevalent virtualization platform, is very frequently the infrastructure used for OpenShift deployments. VMware recently announced the general availability of vSphere 7, the newest version of the hypervisor, and its ecosystem, adding many new features and capabilities for virtual machines and administrators.

As you would expect with a new release, especially one that is a major version change, there are many additions to the APIs and other aspects, which OpenShift heavily depends on for containers to access resources. Working with VMware, we have done some preliminary testing to verify that OpenShift 4.3 and 4.4 successfully deploy to vSphere 7, with and without the vSphere 7 with Kubernetes add-on enabled.

This blog outlines how you can get started with OpenShift 4.4 on VMware vSphere 7. Today, this is in unsupported developer preview with generally available support planned for an upcoming release.

vSphere 7

With the newest version of VMware’s vSphere, there are several new capabilities that are beneficial to OpenShift 4 workloads:

• New DRS Scheduler - A new version of the Dynamic Resource Scheduler (DRS) focuses on application workload responsiveness, rather than virtual machine resource balancing. OpenShift containerized workloads greatly benefit from this new granularity and resource balancing.
• Assignable Hardware - Assignable Hardware in vSphere 7 provides a flexible mechanism to assign hardware accelerators to workloads such as GPU or FPGA hardware. This mechanism identifies the hardware accelerator by attributes of the device and, optionally, tags. This allows for a level of abstraction of the PCIe device and workloads that require hardware acceleration to be dynamically scheduled onto ESXi hosts with the appropriate capability and the PCIe device mapped directly up into the virtual machine. This allows an OpenShift cluster to be customized for GPU acceleration but still allows efficient resource allocation and optimization (via DRS).
• Cloud-Native Storage - With vSphere 7, Cloud Native Storage (CNS) in vSphere has been expanded to include support for ReadWriteMany (RWM) volumes through vSAN File Services and support for vVOL datastores, among other day-2 operational enhancements. Integration into the vSphere vCenter Console has also been further enhanced to display mapping OpenShift Persistent volumes directly back to the virtual disks (VMDKS) that contain the application data.

OpenShift on vSphere 7

Deploying OpenShift to vSphere is done in the same way for vSphere 7 as it has been with vSphere 6.5 and 6.7. The integration features, such as dynamic storage provisioning, continue to behave the same way as well. The result is an OpenShift cluster that has three masters and two or more worker nodes, using OpenShiftSDN for pod-to-pod communication, and the in-tree vSphere storage provisioner.

The user provisioned infrastructure (UPI) deployment method, which is sometimes referred to as pre-existing infrastructure, is the recommended way to install OpenShift 4 to vSphere. Full-stack automated deployments, sometimes called installer provisioned infrastructure (IPI), are not yet available when using vSphere as the infrastructure platform for OpenShift 4.

The steps for installing are:

1. Satisfy the prerequisites for DNS and load balancing.
2. Create the Ignition configs.
3. Create the VMs and install Red Hat Enterprise Linux CoreOS, providing the appropriate Ignition file for the node type.
4. Finish configuring the cluster by approving certificate signing requests, providing registry storage, and validating the install was successful.

The image below shows OpenShift 4.3 virtual machines deployed to a vSphere 7 environment that has the vSphere 7 with Kubernetes feature enabled. We can see the vSphere namespace for a tenant and the supervisor cluster’s control plane virtual machines deployed alongside the OpenShift virtual machines.

Here is a video that shows OpenShift 4.4 being deployed, using the user provisioned infrastructure method, to the same vSphere 7 cluster:

VMware Cloud Foundation

VMware Cloud Foundation is an integrated software platform incorporating a full range of software-defined services (vSphere 7, vSAN 7, NSX SDN) into a single lifecycle managed deliverable that can be deployed in an on-premises private cloud or run as a service in a public cloud. VMware Cloud Foundation offers a simple path to the creation of a hybrid cloud environment by using a common platform for private and public clouds, eliminating any problems with compatibility and making for a consistent operational experience. The same tools and processes can be used in public and private clouds. The unified SDDC platform is an enterprise-ready cloud infrastructure for public and private clouds.

Deploying OpenShift 4 to a VCF 4 workload domain provides a solid foundation for hosting your containerized applications that provides compute, storage, and network resources on a single managed infrastructure automatically deployed, managed, and updated alongside traditional virtual machine hosted workloads.

NSX SDN

Using the same SDN (NSX) for OpenShift containerized workloads and virtual machines also derive the following key benefits:

• Performance optimization by avoiding double encapsulation between environments with
  Service type LoadBalancer is realized automatically on demand
• Admin Firewall policy enforced per service, per cluster, or across all clusters
• Distributed Firewall and Distributed Intrusion Detection System per Pod
• Reliable egress source IP address per OCP Project and per Service
• A mix of private and routed subnets per OpenShift Project
• Single pane of glass for OpenShift, Kubernetes, and VM workload
• Network Quality of Service
• Service insertion to redirect traffic between Pods to third-party security appliance
• Visibility and troubleshooting tools like NSX Traceflow, IPFIX, Port Mirroring, vRNI

Working Together for the Future

VMware and Red Hat continue to work together to deliver more integration and capabilities between OpenShift and vSphere. OpenShift’s roadmap not only includes full support for vSphere 7, but also installer-provisioned infrastructure compatibility to enable the simple and rapid deployment of additional OpenShift clusters on-demand. Until then, OpenShift and vSphere Kubernetes guest clusters happily coexist in the same infrastructure without special configuration for vSphere, OpenShift, or the other components of VMware Cloud Foundation.

We continuously work together to address the needs and concerns of our mutual customers. If you have questions, please leave a comment below or reach out to your Red Hat or VMware account team.

For more information, please visit these links:

• Red Hat OpenShift on VMware
• The OpenShift 4.4 documentation
• VMware vSphere 7 documentation
• vSphere Cloud Provider documentation (deprecated)
• vSphere CSI Driver documentation (Preferred for new installs)
• Installing and configuring vSphere CSI Driver on OpenShift 4.3
• vSphere 7 – Improved DRS
• vSphere 7 – Assignable Hardware
• Cloud-Native Storage and vSAN File Services Integration