Creating a ROSA cluster with PrivateLink enabled (custom VPC) and STS

This is a combination of the private-link and sts setup documents to show the full picture

Prerequisites

• AWS CLI
• Rosa CLI v1.1.7
• jq

AWS Preparation

1. If this is a brand new AWS account that has never had a AWS Load Balancer installed in it, you should run the following

Create the AWS Virtual Private Cloud (VPC) and Subnets

For this scenario, we will be using a newly created VPC with both public and private subnets. All of the cluster resources will reside in the private subnet. The public subnet will be used for traffic to the Internet (egress)

Note: If you already have a Transit Gateway (TGW) or similar, you can skip the public subnet configuration

Note: When creating subnets, make sure that subnet(s) are created in availability zones that have ROSA instances types available. If AZ is not “forced”, the subnet is created in a random AZ in the region. Force AZ using the --availability-zone argument in the create-subnet command.

1. Use rosa list instance-types to list the ROSA instance types

2. Use aws ec2 describe-instance-type-offerings to check that your desired AZ supports your desired instance type

   Example using us-east-1, us-east-1b, and m5.xlarge:  

   Result should display INSTANCETYPEOFFERINGS [instance-type] [az] availability-zone if your selected region supports your desired instance type

3. Configure the following environment variables, adjusting for ROSA_CLUSTER_NAME, VERSION and REGION as necessary

4. Create a VPC for use by ROSA

   • Create the VPC and return the ID as VPC_ID

   • Tag the newly created VPC with the cluster name

   • Configure the VPC to allow DNS hostnames for their public IP addresses

   • The new VPC should be visible in the AWS console

     

5. Create a Public Subnet to allow egress traffic to the Internet

   • Create the public subnet in the VPC CIDR block range and return the ID as PUBLIC_SUBNET

   • Tag the public subnet with the cluster name

6. Create a Private Subnet for the cluster

   • Create the private subnet in the VPC CIDR block range and return the ID as PRIVATE_SUBNET

   • Tag the private subnet with the cluster name

   • Both subnets should now be visible in the AWS console

     

7. Create an Internet Gateway for NAT egress traffic

   • Create the Internet Gateway and return the ID as I_GW

   • Attach the new Internet Gateway to the VPC

   • Tag the Internet Gateway with the cluster name

   • The new Internet Gateway should be created and attached to your VPC

     

8. Create a Route Table for NAT egress traffic

   • Create the Route Table and return the ID as R_TABLE

   • Create a route with no IP limitations (0.0.0.0/0) to the Internet Gateway

   • Verify the route table settings

     Example output

   • Associate the Route Table with the Public subnet

     Example output

   • Tag the Route Table with the cluster name

9. Create a NAT Gateway for the Private network

   • Allocate and elastic IP address and return the ID as EIP

   • Create a new NAT Gateway in the Public subnet with the new Elastic IP address and return the ID as NAT_GW

   • Tag the Elastic IP with the cluster name

   • The new NAT Gateway should be created and associated with your VPC

     

10. Create a Route Table for the Private subnet to the NAT Gateway

    • Create a Route Table in the VPC and return the ID as R_TABLE_NAT

    • Loop through a Route Table check until it is created

      Example output!
      

      

    • Create a route in the new Route Table for all addresses to the NAT Gateway

    • Associate the Route Table with the Private subnet

    • Tag the Route Table with the cluster name

Configure the AWS Security Token Service (STS) for use with ROSA

The AWS Security Token Service (STS) allows us to deploy ROSA without needing a ROSA admin account, instead it uses roles and policies to gain access to the AWS resources needed to install and operate the cluster.

This is a summary of the official OpenShift docs that can be used as a line by line install guide.

Note that some commands (OIDC for STS) will be hard coded to US-EAST-1, do not be tempted to change these to use $region instead or you will fail installation.

1. Make you your ROSA CLI version is correct (v1.1.0 or higher)

2. Create the IAM Account Roles

Deploy ROSA cluster

1. Run the rosa cli to create your cluster

   Confirm the PrivateLink set up

2. Create the Operator Roles

3. Create the OIDC provider.

4. Validate The cluster is now installing

   The State should have moved beyond pending and show installing or ready.

5. Watch the install logs

Validate the cluster

Once the cluster has finished installing it is time to validate. Validation when using PrivateLink requires the use of a jump host.

You can create them using the AWS Console or the AWS CLI as depicted below:

1. Option 1: Create a jump host instance through the AWS Console

   • Navigate to the EC2 console and launch a new instance

   • Select the AMI for your instance, if you don’t have a standard, the Amazon Linux 2 AMI works just fine

   • Choose your instance type, the t2.micro/free tier is sufficient for our needs, and click Next: Configure Instance Details

   • Change the Network settings to setup this host inside your private-link VPC

   • Change the Subnet setting to use the private-link-public subnet

   • Change Auto-assign Public IP to Enable

   • Default settings for Storage and Tags are fine. Make the following changes in the 6. Configure Security Group tab (either by clicking through the screens or selecting from the top bar)

     • If you already have a security group created to allow access from your computer to AWS, choose Select an existing security group and choose that group from the list, otherwise, select Create a new security group and continue.

     • To allow access only from your current public IP, change the Source heading to use My IP

   • Click Review and Launch, verify all settings are correct, and follow the standard AWS instructions for finalizing the setup and selecting/creating the security keys.

   • Once launched, open the instance summary for the jump host instance and note the public IP address.

2. Option 2: Create a jumphost instance using the AWS CLI

   • Create an additional Security Group for the jumphost

   • Grab the Security Group Id generated in the previous step

   • Add a rule to Allow the ssh into the Public Security Group

   • (Optional) Create a Key Pair for your jumphost if your have not a previous one

   • Define an AMI_ID to be used for your jump host

     This AMI_ID corresponds an Amazon Linux within the us-east-1 region and could be not available in your region. Find your AMI ID and use the proper ID.

   • Launch an ec2 instance for your jumphost using the parameters defined in early steps:

     This instance will be associated with a Public IP directly.

   • Wait until the ec2 instance is in Running state, grab the Public IP associated to the instance and check the if the ssh port and:

3. Create a ROSA admin user and save the login command for use later

4. Note the DNS name of your private cluster, use the rosa describe command if needed

5. update /etc/hosts to point the openshift domains to localhost. Use the DNS of your openshift cluster as described in the previous step in place of $YOUR_OPENSHIFT_DNS below

6. SSH to that instance, tunneling traffic for the appropriate hostnames. Be sure to use your new/existing private key, the OpenShift DNS for $YOUR_OPENSHIFT_DNS and your jump host IP for $YOUR_EC2_IP

   

7. From your EC2 jump instances, download the OC CLI and install it locally

   • Download the OC CLI for Linux
   • Unzip and untar the binary

8. log into the cluster using oc login command from the create admin command above. ex.

   

9. Check that you can access the Console by opening the console url in your browser.

Cleanup

1. Delete ROSA

2. Watch the logs and wait until the cluster is deleted

3. Clean up the STS roles

   Note you can get the correct commands with the ID filled in from the output of the previous step.

4. Delete AWS resources