ARO - Cross Tenant Provisioning

Summary

There may be situations where you want to create an ARO cluster where the organization has a policy which has a central entity that controls things such as encryption keys or networking components. This is desirable in large enterprises due to separation of concerns and limiting areas of control for groups to a small scope. This does present challenges, as those different groups must be able to integrate with one another. Often times, the integration is difficult, complex, and confusing. This document serves as a way to clear up some of the confusion by walking you through scenarios for cross-tenancy in ARO.

This guide covers the below use case. Feel free to ignore sections not related to your use case:

• Use Encryption Keys that resides within a separate Azure subscription

Environment Setup

We need to set the following inputs related to the tenant where ARO and its networking is being installed before proceeding:

• SCRATCH_DIR - a location from your workstation where temporary files are stored
• ARO_CLUSTER_NAME - the name of the cluster you are creating (also used as the prefix for other resources)
• ARO_PULL_SECRET - the pull secret to use when provisioning the OpenShift cluster
• ARO_RESOURCE_GROUP - the resource group where the ARO cluster will be created
• ARO_LOCATION - the location where the ARO cluster will be created
• ARO_VNET_RESOURCE_GROUP - the resource group where the VNET for the ARO cluster will be created
• ARO_VNET - the VNET where the ARO cluster will be created
• ARO_CONTROL_PLANE_SUBNET - the subnet which hosts the ARO control plane
• ARO_WORKER_SUBNET - the control plane subnet which hosts the ARO worker nodes
• ARO_APP_NAME - the application name which represents the ARO identity which is used to access resources in other tenants.

We need to set the following inputs related to the tenant where the Azure Key Vault exists before proceeding:

• KV_RESOURCE_GROUP - the resource group where the Azure Key Vault will be created
• KV_LOCATION - the location where the Azure Key Vault will be created
• KV_VAULT_NAME - the name of the Azure Key Vault that will be created
• KV_KEY_NAME - the name of the key within the Azure Key Vault that will be created

Feel free to use the sensible defaults below if you need to quickly see how this works:

Cross-Tenant Encryption Setup

This section covers the guide for cross-tenant encryption.

Setup the ARO Tenant

1. Be sure to login the the tenant that will host your ARO cluster:

2. Create the application and store the client ID and client secret of the application. This application represents ARO when it is authenticating against other resources:

3. Create the resource group to host the ARO cluster:

4. Create the managed identity for the application. This will be used as the identity to host the federated credential required for cross-tenant authentication. We will store the id output for use in followon steps:

5. Create the federated credential for the managed identity:

Setup the Key Vault Tenant

This step configures the Key Vault and grants access to the application created in the ARO tenant.

1. Be sure to login the the tenant that will host your Azure Key Vault and Key:

2. Create the service principal with the registered application name. This will link the managed identity to this newly created service account:

3. Create the resource group to host the Azure Key Vault and and the encryption key:

4. Create the Azure Key Vault and store the id for use in followon steps:

5. In order to create keys in the vault, you need to be assigned the Key Vault Crypto Officer role. You can assign yourself the role with the following:

6. Create the encryption key and store the Key URL in a variable:

7. Grant the ability for the ARO identity to use the key:

Create the Disk Encryption Set

1. Be sure to login the the tenant that will host your ARO cluster:

2. Create the disk encryption set which uses the newly created encryption key. Store the id from the output as it will be used in provisioning the cluster:

Create the Networking

1. Create the resource group to host the ARO networking components:

2. Create the VNET for the ARO cluster. We need to store the fully qualified resource ID and use it in the provisioning process:

3. Create the control plane subnet. We need to store the fully qualified resource ID and use it in the provisioning process:

4. Disable network policies on the control plane subnet. This is required for the ARO service to be able to connect to and manage the cluster:

5. Create the worker subnet. We need to store the fully qualified resource ID and use it in the provisioning process:

Provision the ARO Cluster

Finally you can provision the cluster. Keep in mind that your command may be different if you did not use both use cases above. For reference the inputs from each use case are:

• Use Encryption Keys that resides within a separate Azure subscription
  • DES_ID - the fully qualified disk encryption set resource ID which exists in a separate Azure tenant.
• Inputs from network creation
  • ARO_VNET_ID - the fully qualified VNET resource ID which exists in a separate Azure tenant.
  • ARO_CONTROL_PLANE_SUBNET_ID - the fully qualified control plane subnet resource ID which exists in a separate Azure tenant.
  • ARO_WORKER_SUBNET_ID - the fully qualified worker subnet resource ID which exists in a separate Azure tenant.

1. Run the command to create the cluster:

Additional Resources

• Cross Tenant Customer-Managed Keys
• ARO BYO-Key How-To Guide