Running your Red Hat OpenShift Container Platform 4.12 on Oracle Cloud VMware Solution (OCVS)

Red Hat® OpenShift® Container Platform automates installation, upgrades, and lifecycle management of the container stack, including the OS, Kubernetes, and applications on any cloud. It prioritizes security and provides long-term support. Oracle Cloud VMware Solution™ enhances OpenShift manageability by offering a flexible, scalable, and secure infrastructure. It provisions compute, network, and storage on demand, enabling optimal performance and compliance with micro-segmentation and data-at-rest encryption.

This blog will provide design and deployment guidelines for running Red Hat OpenShift on Oracle Cloud VMware Solution.

The solution's infrastructure automation capabilities enable administrators to swiftly deploy, manage, and scale the underlying infrastructure with the agility of cloud computing, aligned with the pace of business.

Running OpenShift on Oracle Cloud VMware Solution delivers the advantages of a modern public cloud built on VMware Software-Defined Data Center architecture:

• Standardized infrastructure approach that is consistent and repeatable
• Automation to eliminate human error and enhance administrative productivity
• Cloud-scale and agility to enable seamless scalability in line with business requirements
• Integration with VMware's well-established networking solution, VMware NSX™
• Integration with VMware's proven storage solution, vSAN™
• Utilization of VMware vSphere® features such as VMware vSphere Distributed Resource Scheduler™ (DRS), vSphere vMotion®, vSphere HA, and more

Kubernetes vSphere CSI Driver

The technologies leveraged in this solution include Oracle Cloud VMware Solution, VMware vSphere, vSAN, NSX Data Center, vSphere CSI for Kubernetes Storage, Red Hat OpenShift Container Platform, and VMware NSX Container Plug-in for OpenShift.

One key component of this solution is Cloud Native Storage (CNS). CNS bridges the gap between vSphere and Kubernetes, allowing for on-demand storage provisioning in a fully automated and scalable fashion. It empowers administrators with visibility into container volumes through the CNS user interface within vCenter. Organizations can streamline infrastructure needs, reduce costs, and enhance operational efficiency by running and managing containers and virtual machines on the same platform.

The main objective of CNS is to leverage vSphere's mature and reliable data path, including vSAN, as a platform for running stateful Kubernetes workloads. CNS enables Kubernetes to access this robust data path, providing high reliability and performance. It also brings an understanding of Kubernetes volume and pod abstractions to vSphere, facilitating seamless integration.

By embracing Cloud Native Storage, organizations can simplify infrastructure needs, reduce costs, and free up valuable resources. Spend less time managing infrastructure and focus more on building applications that deliver tangible business value. To read more about CNS, visit this link.

Red Hat OpenShift Container Platform delivers a comprehensive solution for container orchestration, leveraging Red Hat Enterprise Linux CoreOS for Kubernetes control plane nodes. It supports both Red Hat Enterprise Linux CoreOS and Red Hat Enterprise Linux for worker nodes. OpenShift aligns with the Open Container Initiative (OCI) standards, providing compatibility with container formats and runtimes. The platform undergoes rigorous testing with a wide range of technologies, resulting in a tightly integrated and robust solution. OpenShift includes software-defined networking capabilities and validates compatibility with various networking solutions. Additionally, it validates numerous storage and third-party plug-ins to provide seamless integration and flexibility.

VMware NSX Container Plug-in for OpenShift

VMware NSX Container Plug-in (NCP) enables seamless integration between NSX Data Center and container orchestrators like Kubernetes and container-based PaaS solutions like OpenShift.

NCP consists of a container-based component that communicates with NSX Manager and the OpenShift control plane. It monitors container and resource changes and manages networking resources, including logical ports, switches, routers, and security groups, by utilizing the NSX Policy API.

The NSX CNI plug-in, installed on each OpenShift node, handles container lifecycle events, connects container interfaces to the guest vSwitch, and configures the guest vSwitch to manage container traffic between interfaces and vNIC.

OCVS Installation

Oracle Cloud VMware Solution (OCVS) is deployed within Oracle Cloud Infrastructure (OCI), utilizing the OCI Virtual Cloud Network (VCN) for network setup. The tests that were run in this solution comprised a single SDDC (see architecture below). OCVS Software-Defined Data Center (SDDC) can be provisioned through the Oracle Cloud Console or programmatically using OCI-CLI, SDK bindings, or the OCI Terraform Provider. OCVS is connected to the outside world via OCI VCN, enabling routing to and from various networks and regions. Public internet access can be facilitated through OCI or an existing corporate internet gateway. Routing between VCNs and on-net destinations can be achieved without NAT, and connections with on-premises networks can be established using FastConnect or VPN Connect services. Within OCVS, OCI and NSX environments provide network services and additional services can be deployed as virtual appliances in both OCI and OCVS environments.

Figure 1. Deployment of the OpenShift Container Platform into the SDDC

OpenShift Installation

Figure 2: OpenShift Container Platform installation targets and dependencies

First, obtain a valid Red Hat OpenShift Pull Secret file from the Red Hat Cloud Console for cluster authentication and updates. Note that an air-gap installation method is available for clusters without internet connectivity.

Next, create a new NSX segment called 192.168.200.0/24 with a gateway and network address of 192.168.200.1/24. Use this segment for deploying the OpenShift Cluster nodes.

Deploy a Linux server with DNS and DHCP roles enabled to the new network segment. Configure the DHCP service with an address pool of 192.168.200.100-192.168.200.200.

Create specific Host A DNS records pointing to reserved IP addresses on the same network subnet where the OpenShift nodes are deployed. For example, create records for api.{cluster_name}.{base_domain} and *.apps.{cluster_name}.{base_domain}.

Configure NSX Distributed firewall rules to allow outbound HTTPS access to the public internet from the subnet group 192.168.200.0/24. Additionally, allow DNS requests from the Linux server to an upstream public DNS service.

In a production environment, it is advisable to implement stricter micro-segmentation rules within subnets and for any outgoing traffic.

Lastly, deploy and configure a second Linux server as a bastion host with the necessary tools: OpenShift-Install CLI, OpenShift Client CLI, Kubectl CLI, and an SSH server. Download and install the certificates from the vCenter endpoint by following the instructions provided by your operating system.

Fulfilling these prerequisites will prepare you to deploy the Red Hat OpenShift cluster successfully.

Installing Red Hat OpenShift Using the openshift-install Tool

To validate the Red Hat OpenShift deployment, the "Installer provisioned Installation" method was followed. This approach involves deploying a reference architecture of Red Hat OpenShift on your selected platform, including the necessary infrastructure components.

There are two ways to utilize the openshift-install tool when using this installation method. The first option is to run the wizard, which guides you through various prompts such as domain name, cluster name, vCenter details, authentication, and the pullSecret file.

The second option is to create a YAML file, providing more flexibility and control. This YAML file allows you to specify VM resources for the cluster nodes. We recommend using the latter method if you intend to integrate with NSX. You can initiate it by running the command openshift-install create install-config, which generates an install-config.yaml file for editing or by directly creating the form yourself using the same name.

To initiate the installation wizard or consume an existing install-config.yaml file, simply run the command openshift-install create cluster.

During the installation process, the tool will provide console output, which can be customized by adjusting the log level using the command line argument --log-level={level}.

INFO Consuming Install Config from target directory

INFO Creating infrastructure resources...        

INFO Waiting up to 20m0s (until 9:52AM) for the Kubernetes API at https://api.ocp-ocvs.veducate.local:6443...

INFO API v1.25.4+18eadca up                      

INFO Waiting up to 30m0s (until 10:04AM) for bootstrapping to complete...

INFO Destroying the bootstrap resources...       

INFO Waiting up to 40m0s (until 10:30AM) for the cluster at https://api.ocp-ocvs.veducate.local:6443 to initialize...

INFO Checking to see if there is a route at openshift-console/console...

INFO Install complete!                           

INFO To access the cluster as the system:admin user when using 'oc', run 'export KUBECONFIG=/home/ocp-ocvs/auth/kubeconfig'

INFO Access the OpenShift web-console here: https://console-openshift-console.apps.ocp-ocvs.veducate.local

INFO Login to the console with user: "kubeadmin", and password: "ur6xT-gxmVW-WVUuD-Sd44J"

INFO Time elapsed: 35m16s  

Once the installation is complete, you can interact with the cluster using the provided kubeconfig file or the OpenShift Console UI.

It's important to note that Red Hat and VMware have jointly validated this installation. While it is a fully supported deployment of Red Hat OpenShift on vSphere through OVCS, extra effort has been dedicated to confirming that the OpenShift deployment functions as expected within the OCVS infrastructure and successfully passes the internal conformance tests at Red Hat.

Keep in mind that these instructions apply to any installation method chosen for deploying OpenShift on vSphere. Refer to the "Preparing to install on vSphere" documentation for detailed instructions,.

References:

• Red Hat OpenShift Container Platform 4.12 on Oracle Cloud VMware Solution
• VMware vSphere 
• VMware vSAN
• VMware NSX Data Center
• Red Hat OpenShift Container Platform