Red Hat OpenShift can be deployed on many platforms on many clouds. One solution is to use Azure Red Hat OpenShift (more commonly known as ARO). ARO is an offering by Microsoft Azure that is supported, jointly, by Red Hat and Microsoft.

One of the many features that is provided to you is to use Microsoft’s managed Active Directory service called Azure Active Directory. You can use this service to provide authentication to your ARO installation.

In this blog I will show you how to configure your ARO instance to use Azure Active Directory for logins and share my experience in the hopes that it may help others in the process.

Set up the Requirements

Install or Update Azure CLI

I will be using Mac OSX, so I installed the CLI using the brew utility. You can visit the official documentation page on the Azure site for instructions for other platforms:

$ brew update && brew install azure-cli

NOTE:

Make sure you have permission to create resources in the resource group. I logged in as a global administrator when I was testing this.

Set up the Environment Variables

I set up environment variables for my installation so you can follow along:

$ cat aro-env
LOCATION=centralus. # the location of your cluster
RESOURCEGROUP=aro-rg # the name of the resource group where you want to create your cluster
CLUSTER=poc #cluster-id of the ARO 4 cluster
$ source aro-env

Log In Azure

az login

Your default browser will open the Azure login page.

Create a Resource Group

You need a resource group, so use the CLI to create one:

az group create \
--name $RESOURCEGROUP \
--location $LOCATION

Add DNS Zone

If you don’t have a DNS zone already, you can get one directly from Azure:

• Log in Azure Portal
• Type: "DNS Zones" in the search box on the top and click on "DNS Zones"
• Click "+Add" on the top
• Select the newly created resource group
• Enter your domain
• Select the location
• Create "Review+Create"

NOTE:

If you are using a domain outside of Azure, You will need to add the NS records from the DNS zone from the overview page of the DNS zone to your domain. Also, request an increase of quota from Azure portal. ARO requires a minimum of 40 cores.

Register Resource Provider

az account set --subscription
az provider register -n Microsoft.RedHatOpenShift --wait
az provider register -n Microsoft.Compute --wait
az provider register -n Microsoft.Storage --wait

Create a Virtual Network

az network vnet create \
--resource-group $RESOURCEGROUP \
--name aro-vnet \
--address-prefixes 10.0.0.0/22

Create an Empty Subnet for Master Nodes

az network vnet subnet create \
--resource-group $RESOURCEGROUP \
--vnet-name aro-vnet \
--name master-subnet \
--address-prefixes 10.0.0.0/23 \
--service-endpoints Microsoft.ContainerRegistry

Create an Empty Subnet for Worker Nodes

az network vnet subnet create \
--resource-group $RESOURCEGROUP \
--vnet-name aro-vnet \
--name worker-subnet \
--address-prefixes 10.0.2.0/23 \
--service-endpoints Microsoft.ContainerRegistry

Disable Private Endpoint Policy

az network vnet subnet update \
--name master-subnet \
--resource-group $RESOURCEGROUP \
--vnet-name aro-vnet \
--disable-private-link-service-network-policies true

Once the above steps are done, you do not have to redo the steps if you are going to reuse the names and resources. When I was testing the installation, I deleted and re-created the cluster many times with the same create command as shown later. I did not need to repeat the above steps when re-creating the cluster.

Create a Service Principal

In the process of creating ARO 4, it creates a service principal if it is not explicitly assigned from the create command. To avoid having a random service principal created by the process, I created a service principal for cluster creation and used the same service principal for configuring the Azure Active Directory integration as well:

az ad sp create-for-rbac --role Contributor --name all-in-one-sp

This command will return the “appId” and “password” information of the service principal that we will need for the ARO 4 create command later.

Add API Permission to the Service Principal

• Log in to Azure Portal
• Go to Azure Active Directory
• Click App registrations
• Click "All applications"
• Search for "app-in-one-sp"
• Click "View API permission"
• Click "Add a permission"
• Click "Azure Active Directory Graph"
• Click "Delegated Permissions"
• Check "User.Read"
• Click the "Add permission" button at the bottom.
• Click "Grant admin consent ..."
• A green checkmark is shown under Status as indicated below:

Create Cluster

Please make sure you log in to Azure and environment variables are set.

Information Needed for Creating a Cluster

• Get a copy of the pull secret from cloud.redhat.com. If you do not have a user name created, please just register as a user for free.
• Create an ARO cluster using the following command. Please apply to appropriate values. Some values were used in the example are explained as shown below.
• aro-vnet - the name of virtual network
• master-subnet - the name of master subnet
• worker subnet - the name of worker subnet
• ./pull-secret.txt - the path and pull secret where is located
• aro.ocpdemo.online - custom domain for the cluster

Create ARO With the Existing Service Principal

az aro create \
--resource-group $RESOURCEGROUP \
--name $CLUSTER \
--client-id <service principal application id> \
--client-secret <service principal password> \
--vnet aro-vnet \
--master-subnet master-subnet \
--worker-subnet worker-subnet \
--pull-secret @./pull-secret.txt \
--domain aro.ocpdemo.online

The information from the JSON output of the above command can be useful if you are not familiar with OpenShift 4. You can find your API server IP, API URL, OpenShift console URL, and ingress IP. You will need the API and ingress IP for the next step:

{- Finished ..
"apiserverProfile": {
"ip": "x.x.x.x",
"url": "https://api.aro.ocpdemo.online:6443/",
"visibility": "Public"
...
},
"consoleProfile": {
"url": "https://console-openshift-console.apps.aro.ocpdemo.online/"
},
....
"ingressProfiles": [
{
"ip": "x.x.x.x",
"name": "default",
"visibility": "Public"
}
....

Post ARO Installation

Adding Two A Records for API and *.apps in the DNS Zone

• Log in to Azure Portal
• Go to DNS zone
• Click onto the domain for the ARO cluster
• Click "+ Record Set" on the top menu to create an A record and add values to Name and IP. You will need to repeat this step for both api and *.apps A records.
• Name: api or *.apps
• IP: the apps/ingress IP is from the output of the creation of the ARO
• The below screenshot shows the DNS zone configuration, and two A records were added:

Test ARO Cluster

Getting the Kubeadmin Credential

az aro list-credentials \
--name $CLUSTER \
--resource-group $RESOURCEGROUP

The command returns the kubeadmin credential.

Log in OpenShift Console

Open a browser and go to the OpenShift console or look for "consoleProfile" from the JSON output of ARO creation:

https://console-openshift-console.apps.<DNS domain>/

The login user is kubeadmin and the password is the credential from the last command. Congrats! The ARO installation is completed.

Azure Active Directory Integration

Getting OpenShift CLI

Download OpenShift command line tool from console:

Using openshift CLI, we need the OpenShift Command Lind Interface (CLI). Once you download, extract the CLI and add it to the PATH. You can move on to the next step.

Log in to ARO Via OC CLI

oc login -u kubeadmin -p <password> https://api.<DNS domain>:6443/

Getting OAuth Callback URL

$ oauthCallBack=`oc get route oauth-openshift -n openshift-authentication -o jsonpath='{.spec.host}'`

$ oauthCallBackURL=https://$oauthCallBack/oauth2callback/AAD

$ echo $oauthCallBackURL

NOTE:

AAD is the name of the identity provider when configuring OAuth on OpenShift

Add the OAuth Callback URL to the Same Service Principal

• Go to Azure Active Directory
• Click App registration
• Click on "all-in-one-sp" under all applications
• Under Overview, click right top corner link for "Add a Redirect URI"
• Click "Add a platform"
• Click Web Application from the list of Configure platforms
• Enter the value of the $oauthCallBackURL from the previous step to the "Redirect URIs"
• Click configure

Create a manifest file:

$ cat > manifest.json<< EOF
[{ "name": "upn",
"source": null,
"essential": false,
"additionalProperties": []
},
{ "name": "email",
"source": null,
"essential": false,
"additionalProperties": []
}]
EOF

Update service principal with the manifest:

az ad app update \
--set optionalClaims.idToken=@manifest.json \
--id <Service Principal appId>

Create a secret to store service principal's password:

oc create secret generic openid-client-secret-azuread \
--namespace openshift-config \
--from-literal=clientSecret=<service principal password>

Create an OAuth Configuration

apiVersion: config.openshift.io/v1
kind: OAuth
metadata:
  name: cluster
spec:
  identityProviders:
  - name: AAD
    mappingMethod: claim
    type: OpenID
    openID:
      clientID: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
      clientSecret:
        name: openid-client-secret-azuread
      extraScopes:
      - email
      - profile
      extraAuthorizeParameters:
        include_granted_scopes: "true"
      claims:
        preferredUsername:
        - email
        - upn
        name:
        - name
        email:
        - email
      issuer: https://login.microsoftonline.com/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx

NOTE:

• The clientID is the AppId of your registered application.
• Issuer URL is https://login.microsoftonline.com/<tenant id>.
• The clientSecret is using the secret (openid-client-secret-azuread) that you created from the previous step.

Alternatively, you can obtain the clientID and tenant id from Azure Portal:

• Log in Azure Portal
• Click Home
• Click Azure Active Directory
• Click App registrations on the left menu
• Click all applications tab
• Type the application that you just created in the search area
• Click onto the application (my application is all-in-one-sp)
• Under Overview, the information is shown as "Application (client) ID" and Directory (tenant) ID" as in the image below.

Update OpenShift OAuth Configuration

oc apply -f openid.yaml

Login OpenShift console via AAD

It will redirect you to the Azure login page

Delete Cluster

Please make sure you source the environment variables before running this command:

az aro delete --resource-group $RESOURCEGROUP --name $CLUSTER

Troubleshoot

Tip No. 1: If you are getting an error, you can log in as kubeadmin and check the logs from oauth-openshift pods under openshift-authentication project.

Tip No. 2: if you are creating a newly registered application to try on an existing cluster for configuring AAD, make sure you clean up the user and identity.

Special thanks to Christian Hernandez