Using Azure Container Registry in Private ARO clusters

This guide describes how configure and deploy an Azure Container Registry, limiting the access to the registry and connecting privately from a Private ARO cluster, eliminating exposure from the public internet.

You can limit access to the ACR by assigning virtual network private IP addresses to the registry endpoints and using Azure Private Link .

Network traffic between the Private ARO cluster and the registry’s private endpoints traverses the virtual network and a private link on the Microsoft backbone network, eliminating exposure from the public internet.

NOTE: If you are interested in deploy and integrate an ACR with a public endpoint and connect them into an ARO cluster follow the How-to Use ACR with ARO guide .

Prepare your ARO cluster

1. Deploy a Private ARO cluster

2. Set some environment variables

Create ACR and restrict the access using Private Endpoint

You can limit access to the ACR instance by assigning virtual network private IP addresses to the registry endpoints and using Azure Private Link.

Network traffic between the clients on the virtual network and the registry’s private endpoints traverses the virtual network and a private link on the Microsoft backbone network, eliminating exposure from the public internet. Private Link also enables private registry access from on-premises through Azure ExpressRoute private peering or a VPN gateway.

1. Register the resource provider for Azure Container Registry in your subscription:

1. Create PrivateEndpoint-subnet for allocate the ACR PrivateEndpoint resources (among others):

NOTE: Disable network policies such as network security groups in the subnet for the private endpoint it’s needed for the integration with Private Endpoint in this scenario.

1. Create the Azure Container Registry disabling the public network access for the container registry:

1. Create a private Azure DNS zone for the private Azure container registry domain:

NOTE: To use a private zone to override the default DNS resolution for your Azure container registry, the zone must be named privatelink.azurecr.io.

1. Associate your private zone with the virtual network:

1. Get the resource ID of your registry:

1. Create the registry’s private endpoint in the virtual network:

1. Create a DNS zone group for a private endpoint in Azure Container Registry (ACR):

1. Query the Private Endpoint for the Network Interface ID:

1. Get the FQDN of the ACR:

1. Get the Private IP address of the ACR:

1. You can nslookup the FQDN to check that the record it’s propagated properly, and answers with the privatelink one:

1. Get the Username and Password for login to the ACR instance:

1. Try to login with podman or docker to the registry outside of the vNET:

NOTE: you will receive an error, that it’s what we’re expecting, because the access to the ACR it’s restricted outside of the vNET (peering or VPN/ER needs to be used).

1. Get (and save) the ARO_URL and the KUBEADMIN password:

Automation with Terraform (Optional)

If you want to deploy everything on this blog post automated, clone the rh-mobb terraform-aro repo and deploy it:

Testing the Azure Container Registry from the Private ARO cluster

Once we have deployed the ACR, we need to test the ACR instance deployed, and limited the access only from within the vNET (or using peering, VPN or ExpressRoute connectivity).

1. SSH to the JUMPHOST to be able to test and push a example image:

1. Inside of the JUMPHOST (within the vNET) install oc and docker/podman:

1. Login to the registry (this time should work):

1. Push an example image to the ACR:

1. Login to the Private ARO cluster and create a test namespace:

1. Create the Kubernetes secret for storing the credentials to access the ACR inside of the ARO cluster:

1. Link the secret to the service account:

1. Deploy an example app using the ACR container image pushed in the previous step:

1. After a couple of minutes, check the status of the pod:

It should work, deploying the container image in the Private ARO cluster.