User Workload Monitoring on Azure Red Hat OpenShift

In Azure Red Hat OpenShift (ARO) Monitoring for User Defined Projects is disabled by default. Follow these instructions to enable it.

Enabling

See docs for more indepth details.

1. Check the cluster-monitoring-config ConfigMap object

2. Enable User Workload Monitoring by doing one of the following

   If the data.config.yaml is not {} you should edit it and add the enableUserWorkload: true line manually.

   Otherwise if its {} then you can run the following command safely.

3. Create a config for User Workload Monitoring to set retention and

   This will configure the user workload instance to have PVC storage and will set basic data retention values. Feel free to edit it to suit your needs. Remember if you’re going to have PVCs enabled they are tied to an AZ, to for a multi-AZ cluster you should ensure you have at least 2 workers per AZ so that they can failover.

4. Deploy an example application with a service monitor resource

5. Wait a few minutes and then check your cluster metrics.

   1. Switch to Developer mode
   2. Change the Project to ns1
   3. Click the Monitoring button
   

Grafana

1. Create a Project for the Grafana Operator + Application

2. Install the Grafana Operator (or via the OperatorHub in the GUI)

3. Once the Grafana Operator is running create a Grafana Instance

4. Once the instance has been created you should be able to log in by getting the route and using the admin user/pass from above.

   The output should look like

   Copy and paste the host into your browser and log in to verify its working.

5. Grant the grafana instance access to cluster-metrics

6. Save the service accounts bearer token as a variable

7. Create a datasource to access the Thanos Querier

8. Add system dashboards to Grafana

The dashboards.yaml file was created by running the script generate-dashboards.sh which fetches the dashboard json files from the openshift-monitoring namespace.