Protect your ROSA cluster with AWS Transit GW

A systems administrator must make numerous choices on the infrastructure environment that underlies the applications supported by a company. Undoubtedly, one of the most challenging issues is choosing between staying in a traditional environment and moving to the cloud, regardless of the cloud provider.

I can assist you in making the challenging decision to move your workloads to the cloud while maintaining a minimum level of security.

This post covers the choices for using OpenShift Dedicated as the platform for your apps. The cluster, its management interface, and any exposed applications must first be made public or private. It is crucial to emphasize that making anything public or private is irrelevant to its integrity.

Options for connecting to the private cluster

When you make your cluster private, no one can access it. To achieve this, connect the cluster's VPC to an AWS VPC under your account for complete traffic control.

Amazon offers three different kinds of objects for the connection:

• VPC Peering
• Direct Connect
• Transit Gateway

Networks can peer with one another through VPC, similar to being connected by a LAN-to-LAN link.

The alternative approach is Direct Connect, which uses a secure connection that functions like a VPN between a managed cluster and a customer environment.

Finally, the Transit Gateway is the main topic of this article.

An AWS network object called the Transit Gateway enables routing between the cluster's VPC Network and other VPCs in the cloud.

Creating objects

First, create the necessary objects for the private cluster.

The following steps will be required to link your private cluster to an AWS environment using Transit Gateway:

1. Create the Private VPC for the Cluster
2. Create a Private Subnet for the Egress VPC
3. Create a Private ROSA cluster
4. Create the public VPC that will connect the cluster to the Internet and call it Egress VPC
5. Allow DNS between VPCs
6. Create subnets and extra resources

Create private VPC - CLUSTER VPC

If this is a new subscription, do the next step. Otherwise, skip it.

$ aws iam create-service-linked-role --aws-service-name "elasticloadbalancing.amazonaws.com"

export VERSION=4.12.0 \
       ROSA_CLUSTER_NAME=myprivate-rosa \
       AWS_DEFAULT_REGION=us-east-1

VPC_ID_1=`aws ec2 create-vpc --cidr-block 10.0.0.0/16 | jq -r .Vpc.VpcId`

aws ec2 create-tags --resources $VPC_ID_1 --tags Key=Name,Value=rosa_intranet_vpc

Create a private subnet based on the Private VPC

The cluster will reside on this VPC.

ROSA_PRIVATE_SUBNET=`aws ec2 create-subnet --vpc-id $VPC_ID_1 --cidr-block 10.0.0.0/17 | jq -r .Subnet.SubnetId`
aws ec2 create-tags --resources $ROSA_PRIVATE_SUBNET --tags Key=Name,Value=intranet-pvt
echo $ROSA_PRIVATE_SUBNET

Create ROSA cluster - Single Zone

Deploy your ROSA private cluster using the Red Cloud OpenShift Cluster Manager console or via the CLI sample below:

rosa create cluster --private-link --cluster-name=$ROSA_CLUSTER_NAME [--machine-cidr=10.0.0.0/16] --subnet-ids=$ROSA_PRIVATE_SUBNET

IMPORTANT: The ROSA CLI sample requires the creation of the prerequisites described in Red Hat Official Documentation.

Create the Egress VPC

VPC_ID_2=`aws ec2 create-vpc --cidr-block 172.100.0.0/20 | jq -r .Vpc.VpcId`
echo $VPC_ID_2
aws ec2 create-tags --resources $VPC_ID_2 --tags Key=Name,Value=egress_vpc

Set up DNS

aws ec2 modify-vpc-attribute --vpc-id $VPC_ID_1 --enable-dns-hostnames
aws ec2 modify-vpc-attribute --vpc-id $VPC_ID_2 --enable-dns-hostnames

Private subnet for the Egress VPC

TIP: We created the Egress VPC and its subnets in the example. You can skip these tasks if you already have this underlying infrastructure.

EGRESS_PRIVATE_SUBNET=`aws ec2 create-subnet --vpc-id $VPC_ID_2 --cidr-block 172.100.0.0/17 | jq -r .Subnet.SubnetId`
aws ec2 create-tags --resources $EGRESS_PRIVATE_SUBNET --tags Key=Name,Value=egress-pvt
echo $EGRESS_PRIVATE_SUBNET

Public subnet for the Egress VPC

EGRESS_PUBLIC_SUBNET=`aws ec2 create-subnet --vpc-id $VPC_ID_2 --cidr-block 172.100.128.0/17 | jq -r .Subnet.SubnetId`
aws ec2 create-tags --resources $EGRESS_PUBLIC_SUBNET --tags Key=Name,Value=egress-public
echo $EGRESS_PUBLIC_SUBNET

Create the Internet Gateway for the Egress VPC

The Internet Gateway is the AWS object responsible for creating a way for the VPC to access the Internet. To manage the security of your clusters, the recommendation is to limit Internet output to the VPC environment, where network traffic from your other environments is regulated. We are creating the networks from each cluster's perspective and the client-controlled environment's perspective, as this is a proof of concept.

INTERNETGW=`aws ec2 create-internet-gateway | jq -r .InternetGateway.InternetGatewayId`
echo $INTERNETGW
aws ec2 create-tags --resources $INTERNETGW --tags Key=Name,Value=igw-rosa

Attach the Internet Gateway to the Egress VPC

After you create the Internet Gateway, attach it to the Egress VPC.

aws ec2 attach-internet-gateway --vpc-id $VPC_ID_2 --internet-gateway-id $INTERNETGW

Create a NAT Gateway for the Public Egress Subnet

Create a public egress subnet to allow egress traffic thru the egress public subnet only. Associate an Elastic IP to set its identity.

ELASTICIP=`aws ec2 allocate-address --domain vpc | jq -r .AllocationId`
echo $ELASTICIP
NAT_GATEWAY=`aws ec2 create-nat-gateway --subnet-id $EGRESS_PUBLIC_SUBNET --allocation-id $ELASTICIP | jq -r .NatGateway.NatGatewayId`
echo $NAT_GATEWAY
aws ec2 create-tags --resources $ELASTICIP --resources $NAT_GATEWAY --tags Key=Name,Value=egress_nat_public

Create AWS Transit Gateway

Create a Transit Gateway to attach two VPCs.

TRANSITGW=`aws ec2 create-transit-gateway | jq -r .TransitGateway.TransitGatewayId` 
echo $TRANSITGW
aws ec2 create-tags --resources $TRANSITGW --tags Key=Name,Value=rosa-transit-gateway

Create an attachment to the private subnet from the private CLUSTER VPC to the Transit Gateway

The Transit Gateway starts in the pending state. Wait a few minutes until it displays the available state. After that, create a Transit Gateway VPC attachment on the private VPC with the private subnet.

TRANSITGW_A_RPV=`aws ec2 create-transit-gateway-vpc-attachment --transit-gateway-id $TRANSITGW --vpc-id $VPC_ID_1 --subnet-ids $ROSA_PRIVATE_SUBNET | jq -r .TransitGatewayVpcAttachment.TransitGatewayAttachmentId`
echo $TRANSITGW_A_RPV

aws ec2 create-tags --resources $TRANSITGW_A_RPV --tags Key=Name,Value=transit-gw-intranet-attachment

Create an attachment to a private subnet from the Egress VPC to the Transit Gateway

Create a Transit Gateway:

TRANSITGW_A_EPV=`aws ec2 create-transit-gateway-vpc-attachment --transit-gateway-id $TRANSITGW --vpc-id $VPC_ID_2 --subnet-ids $EGRESS_PRIVATE_SUBNET | jq -r .TransitGatewayVpcAttachment.TransitGatewayAttachmentId`
echo $TRANSITGW_A_EPV
aws ec2 create-tags --resources $TRANSITGW_A_EPV --tags Key=Name,Value=transit-gw-egress-attachment

Create an Egress Gateway route

Discover the default Transit Gateway's route table ID:

TRANSITGW_D_RT=`aws ec2 describe-transit-gateways --transit-gateway-id $TRANSITGW | jq -r '.TransitGateways | .[] | .Options.AssociationDefaultRouteTableId'`
echo $TRANSITGW_D_RT

aws ec2 create-tags --resources $TRANSITGW_D_RT --tags Key=Name,Value=transit-gw-rt

Create a static route for Internet traffic to go to the Egress VPC

aws ec2 create-transit-gateway-route --destination-cidr-block 0.0.0.0/0 --transit-gateway-route-table-id $TRANSITGW_D_RT --transit-gateway-attachment-id $TRANSITGW_A_EPV

Provide the main route table to the private VPC

ROSA_VPC_MAIN_RT=`aws ec2 describe-route-tables --filters 'Name=vpc-id,Values='$VPC_ID_1'' --query 'RouteTables[].Associations[].RouteTableId' | jq .[] | tr -d '"'`
echo $ROSA_VPC_MAIN_RT

aws ec2 create-tags --resources $ROSA_VPC_MAIN_RT --tags Key=Name,Value=rosa_main_rt 

Provide the main route table to the Egress VPC

EGRESS_VPC_MAIN_RT=`aws ec2 describe-route-tables --filters 'Name=vpc-id,Values='$VPC_ID_2'' --query 'RouteTables[].Associations[].RouteTableId' | jq .[] | tr -d '"'`
echo $EGRESS_VPC_MAIN_RT

Create a private route table in egress VPC

EGRESS_PRI_RT=`aws ec2 create-route-table --vpc-id $VPC_ID_2 | jq -r .RouteTable.RouteTableId`
echo $EGRESS_PRI_RT

aws ec2 associate-route-table --route-table-id $EGRESS_PRI_RT --subnet-id $EGRESS_PRIVATE_SUBNET

Create a NAT gateway route

Create a route in the egress private route table for all addresses to the NAT gateway.

aws ec2 create-route --route-table-id $EGRESS_PRI_RT --destination-cidr-block 0.0.0.0/0 --gateway-id $NAT_GATEWAY 

Create a route in the egress VPC's main route table for all addresses going to the Internet gateway.

aws ec2 create-route --route-table-id $EGRESS_VPC_MAIN_RT --destination-cidr-block 0.0.0.0/0 --gateway-id $INTERNETGW 

Create a route in the egress VPC's main route table to direct addresses in the OpenShift Service on AWS private VPC to the transit gateway.

aws ec2 create-route --route-table-id $EGRESS_VPC_MAIN_RT --destination-cidr-block 172.100.0.0/20 --gateway-id $TRANSITGW 

Create a route in the OpenShift Service on AWS private route table to direct all its addresses to the transit gateway.

aws ec2 create-route --route-table-id $ROSA_VPC_MAIN_RT --destination-cidr-block 0.0.0.0/0 --gateway-id $TRANSITGW 

Conclusion

After creating all objects related to the Transit Gateway and associating the static routes and route tables to the desired networks, you can control the traffic between a private cluster to subnets inside the customer's VPCs. This approach can route which networks (more than one, as shown) can access the infrastructures, even in the cloud or on-premises.

In this example, I showed how to create anything from VPCs and private clusters to route tables for various networks. You may already have an Internet Gateway and NAT Gateway established in your Amazon environment. In that case, you can deploy a Transit Gateway and its attachments.

Remember that at the conclusion, you will have a diagram that resembles the figure below to use in your documentation.

References

• https://docs.openshift.com/rosa/rosa_install_access_delete_clusters/rosa-aws-privatelink-creating-cluster.html
• https://aws.amazon.com/blogs/containers/red-hat-openshift-service-on-aws-private-clusters-with-aws-privatelink/
• https://aws.amazon.com/blogs/containers/red-hat-openshift-service-on-aws-private-clusters-with-aws-privatelink/
• https://access.redhat.com/documentation/en-us/openshift_dedicated/4/html-single/installing_accessing_and_deleting_openshift_dedicated_clusters/index