Using the AWS Cloud Watch agent to publish metrics to CloudWatch in ROSA

This document shows how you can use the AWS CloudWatch Agent to scrape Prometheus endpoints and publish metrics to CloudWatch in a Red Hat OpenShift Service on AWS (ROSA) cluster.

It pulls from the AWS documentation for installing the CloudWatch Agent to Kubernetes and publishes metrics for the Kubernetes API Server and provides a simple dashboard to view the results.

Currently the AWS CloudWatch Agent does not support pulling all metrics from the Prometheus federated endpoint, but the hope is that when it does we can ship all cluster and user workload metrics to AWS CloudWatch.

Prerequisites

1. A Red Hat OpenShift Service on AWS (ROSA) cluster
2. The OpenShift CLI (oc)
3. The jq command-line interface (CLI)
4. The Amazon Web Services (AWS) CLI (aws)

Setting up your environment

1. Ensure you are logged into your cluster with the OpenShift CLI (oc) and your AWS account with the AWS CLI (aws).

2. Configure the following environment variables:

3. Ensure all fields output correctly before moving to the next section:

Preparing your AWS account

1. Create an IAM role trust policy for the CloudWatch Agent service account to use:

2. Create an IAM role for the CloudWatch Agent to assume:

3. Attach the AWS-managed CloudWatchAgentServerPolicy IAM policy to the IAM role:

Deploy the AWS CloudWatch Agent

1. Create a project for the AWS CloudWatch Agent:

2. Create a ConfigMap with the Prometheus CloudWatch Agent config:

3. Create a ConfigMap for the Prometheus scrape config:

4. Create a service account for the CloudWatch Agent to use and annotate it with the IAM role we created earlier:

5. Create a cluster role and role binding for the service account:

6. Allow the CloudWatch Agent to run with the anyuid security context constraint:

7. Deploy the CloudWatch Agent pod:

8. Verify the CloudWatch Agent pod is Running:

   Example output

Create Sample Dashboard in AWS CloudWatch

1. Download the Sample Dashboard

2. Update the Sample Dashboard

3. Browse to https://console.aws.amazon.com/cloudwatch

4. Create a Dashboard, and name it “Kubernetes API Server”

5. Click Actions and View/edit source

6. Run the following command and copy the JSON output into the text area:

7. After 5-10 minutes, view the dashboard and see the data flowing into CloudWatch: