Integrating Azure ARC with ARO

This document explain how to integrate ARO cluster with Azure Arc-enabled Kubernetes. When you connect a Kubernetes/OpenShift cluster with Azure Arc, it will:

• Be represented in Azure Resource Manager with a unique ID
• Be placed in an Azure subscription and resource group
• Receive tags just like any otherAzure resource

Azure Arc-enabled Kubernetes supports the following scenarios for connected clusters:

• Connect Kubernetes running outside of Azure for inventory, grouping, and tagging.
• Deploy applications and apply configuration using GitOps-based configuration management.
• View and monitor your clusters using Container Insights.
• Enforce threat protection using Microsoft Defender for Kubernetes.
• Apply policy definitions using Azure Policy for Kubernetes.
• Use Azure Active Directory for authentication and authorization checks on your cluster

Prerequisites

• a public ARO cluster
• azure cli
• oc cli
• An identity (user or service principal) which can be used to log in to Azure CLI and connect your cluster to Azure Arc.

Enable Extensions and Plugins

Install the connectedk8s Azure Cli extension of version >= 1.2.0

Register providers for Azure Arc-enabled Kubernetes. Registration may take up to 5 minutes.

Connect an existing ARO cluster

Make sure you are logged into your ARO cluster

Run the following command:

After running the commnad. grant the following permissions and restart kube-aad-proxy pod

Wait for a few mins and you will see all the pods in azure-arc namespace running

This commands take about 5 mins to complete. Upon the completion of the command you should see the following output and your cluster under Kubernetes - Azure Arc service in Azure Portal

To check the status of clusters connected to Azure ARC, run the following command

Enable Resource Viewing

In order to see ARO resource inside Azure Arc, you need to create a service account and provide it to Azure Arc.

Copy the token, goto Azure portal and select your cluster under “Kubernetes - Azure Arc” Select Namespaces from the left side menu and paste the token in “Service account bearer token” input field.

Now you can see all of your ARO resources inside ARC UI. you can see the following resources inside Azure ARC portal:

• Namespaces
• Workloads
• Services and Ingress
• Storage
• Configurations

Enable Container Insights

Follow the Azure documentation to Enable Container insights . Be sure to use “Arc-enabled cluster with ARO” command for ARO-specific instructions.

Access Secrets from Azure Key Vault

The Azure Key Vault Provider for Secrets Store CSI Driver allows for the integration of Azure Key Vault as a secrets store with a Kubernetes cluster via a CSI volume. For Azure Arc-enabled Kubernetes clusters, you can install the Azure Key Vault Secrets Provider extension to fetch secrets.

Install extension

Validate the extension installation

Create or Select an Azure Key Vault

Provide identity to access Azure Key Vault

Currently, the Secrets Store CSI Driver on Arc-enabled clusters can be accessed through a service principal.

First, create the service principal and set an access policy to allow it to get secrets from the keyvault.

Then, store the service principal’s credentials as a secret in your cluster.

Create a SecretProviderClass with the following YAML, filling in your values for key vault name, tenant ID, and objects to retrieve from your AKV instance

Create a pod with the following YAML, filling in the name of your identity

Validate the secrets

After the pod starts, the mounted content at the volume path specified in your deployment YAML is available.

Monitor ARO cluster against Goverance Policies

Azure Policy extends Gatekeeper v3, an admission controller webhook for Open Policy Agent (OPA), to apply at-scale enforcements and safeguards on your clusters in a centralized, consistent manner. Azure Policy makes it possible to manage and report on the compliance state of your Kubernetes clusters from one place. The add-on enacts the following functions:

• Checks with Azure Policy service for policy assignments to the cluster.
• Deploys policy definitions into the cluster as constraint template and constraint custom resources.
• Reports auditing and compliance details back to Azure Policy service.

Azure policy plugin is enabled when you connect your ARO cluster with Azure ARC.

you can click on go to Azure Policies to look at the policies assigned to your cluster, check their status and attach more policies.