Connecting managed clusters with Submariner in Red Hat Advanced Cluster Management for Kubernetes

In this blog, we introduce the new technology preview integration between Submariner and Red Hat Advanced Cluster Management for Kubernetes that is available in version 2.2. This integration can enable direct networking between pods and services across the managed Red Hat OpenShift Container Platform clusters in the hybrid clouds, including on-premises and public clouds like Amazon Web Services, Google Cloud Platform, IBM Cloud, and Microsoft Azure.

Red Hat Advanced Cluster Management for Kubernetes provides end-to-end visibility and control to manage your Kubernetes clusters, and controls your application lifecycle across the hybrid clouds. There is a unique challenge with enabling the workloads that are deployed on the different clusters to communicate with each other when they are managed by Red Hat Advanced Cluster Management.

Submariner, an open source tool that is built to connect overlay networks of different Kubernetes clusters using encrypted VPN tunnels, provides a solution to this challenge. Red Hat Advanced Cluster Management leverages Submariner to provide the network infrastructure over multiple OpenShift Container Platform clusters across the cloud providers, data centers, and regions. The applications or services that are deployed on different OpenShift Container Platform clusters in a ManagedClusterSet, which is a group of managed clusters, can connect with each other securely.

Let's start our new feature journey:

In this example, we provide the steps for you to complete these high-level steps to configure your Submariner connection:

1. Provision two managed clusters on Amazon Web Services.
2. Deploy Submariner on each managed cluster.
3. Deploy an application to verify the cluster connections.

Prerequisites

You need the following things configured in your environment:

You need the following things configured in your environment:

• A Red Hat Advanced Cluster Management hub cluster.
• A ManagedClusterSet that is created on our Red Hat Advanced Cluster Management hub cluster. We created it with the following command:

  cat << EOF | kubectl apply -f -
  apiVersion: cluster.open-cluster-management.io/v1alpha1
  kind: ManagedClusterSet
  metadata:
    name: submariner
  EOF
  

  For more information about creating a ManagedClusterSet, see the documentation for Creating a ManagedClusterSet in the Red Hat Advanced Cluster Management documentation.

Provision our managed clusters on Amazon Web Services

Complete the following steps to provision your clusters:

1. Log into the Red Hat Advanced Cluster Management hub cluster console.
2. From the navigation menu, navigate to Automate infrastructure > Clusters.
3. On the Clusters page, click Add Cluster.
4. Select Create a cluster.
5. Enter a name for the cluster. In this example, the name of the first cluster is cluster1.
6. Select Amazon Web Services for the infrastructure platform.
7. Specify a Release image. Note: The minimum required OpenShift Container Platform version is 4.4.
8. Click Create.
9. Repeat the previous steps to create a second cluster. In this example, the name of the second cluster is cluster2. Note: The two cluster CIDRs (ServiceCIDR and ClusterCIDR) cannot overlap. To ensure this, configure the networking options when you create cluster2 and set the Cluster network CIDR and Service network CIDR manually. In this example, the values are 10.132.0.0/14 and 172.31.0.0/16, respectively.

Deploy Submariner on the managed clusters

After the two clusters are created, complete the following steps on the Red Hat Advanced Cluster Management hub cluster:

1. Create a SubmarinerConfig resource in each managed cluster namespace to prepare the Submariner cluster environment with the following commands:
   • For cluster1:

   cat << EOF | oc apply -f -
   apiVersion: submarineraddon.open-cluster-management.io/v1alpha1
   kind: SubmarinerConfig
   metadata:
     name: subconfig
     namespace: cluster1
   spec:
     credentialsSecret:
        name: cluster1-aws-creds
    EOF
   

   • For cluster2:

   cat << EOF | oc apply -f -
   apiVersion: submarineraddon.open-cluster-management.io/v1alpha1
   kind: SubmarinerConfig
   metadata:
     name: subconfig
     namespace: cluster2
   spec:
     credentialsSecret:
        name: cluster2-aws-creds
   EOF
   

2. Label the managed clusters to deploy the Submariner on the managed cluster with following command:

   oc label managedclusters cluster1 "cluster.open-cluster-management.io/clusterset=submariner" --overwrite
   oc label managedclusters cluster1 "cluster.open-cluster-management.io/submariner-agent=true" --overwrite
   oc label managedclusters cluster2 "cluster.open-cluster-management.io/clusterset=submariner" --overwrite
   oc label managedclusters cluster2 "cluster.open-cluster-management.io/submariner-agent=true" --overwrite
   

After the Submariner is deployed, you can find the Submariner pods in submariner-operator namespace on each managed cluster.

Deploy an application to verify the managed cluster connections

You can use an application to verify the managed cluster connections.

The application in this example consists of a web frontend, redis-leader service for storage, and redis-follower service for storage backup. In this example, we use a Red Hat Advanced Cluster Management application to deploy the frontend and redis-leader service to cluster1. We then deploy the redis-follower service to cluster2. To make this application work, the redis-leader service and redis-follower service should be able to access each other, so we use the serviceexports.multicluster.x-k8s.io API to export them to each managed cluster.

Complete the following steps to deploy your application:

1. Log into to Red Hat Advanced Cluster Management hub cluster console.
2. From the navigation menu, navigate to Manage applications.
3. On the Applications page, click Create application.
4. Enter the application name and namespace.
5. Select Git repository.
6. Enter the application Git URL. For this example, it is https://github.com/skeeey/acm-demo-app.
7. Select the main branch and the guestbook path.
8. Enter the managed cluster cluster1 label name=cluster1. This selects the managed cluster cluster1 to deploy to the application frontend.
9. Repeat steps 2-6.
10. Select the main branch and the redis-leader path.
11. Enter the managed cluster cluster1 label name=cluster1. This selects the managed cluster cluster1 to deploy to the application redis-leader service.
12. Repeat steps 2-6.
13. Select the main branch and the redis-follower path.
14. Enter the managed cluster cluster2 label name=cluster2. This selects the managed cluster cluster2 to deploy to application the redis-follower service.

After the application is deployed, you can access the application with a browser.

Conclusion

Throughout the blog, we outlined a flexible solution to connect the workloads on multiple OpenShift Container Platform clusters. Although this is a tech preview feature, you can try it out to see how easy it is to get started with this new feature.

We continue to add and enhance features to make it more powerful and easy to use, which you can view in our GitHub repository Submariner-addon. Join the open-cluster-management.io community and contribute your use cases and scenarios related to multicluster networking.