Red Hat Advanced Cluster Management for Kubernetes (RHACM) observability service provides many ways to help you gain insight about your managed clusters, and optimize your managed clusters. This information can save costs and prevent unnecessary events. By default, observability service is included with the product installation, but not enabled. Due to the requirement for persistent storage, the observability service is not enabled by default. In this blog, I introduce you to how to use Red Hat OpenShift Data Foundation as stable object stores to store your metric data.

Red Hat OpenShift Data Foundation is software-defined storage for containers. Engineered as the data and storage services platform for Red Hat OpenShift, Red Hat OpenShift Data Foundation helps teams develop and deploy applications quickly and efficiently across clouds.

In this blog, I provide the steps for you to complete these high-level steps to configure your RHACM observabilty service with Red Hat OpenShift Data Foundation:

  1. Deploy Red Hat OpenShift OpenShift Data Foundation.
  2. Enable Red Hat Advanced Cluster Management observabilty service with Red Hat OpenShift Data Foundation.

Prerequisites

You need the following things configured in your environment:

  • Configure oc (ver. 4.3+) and kubectl (ver. 1.16+) to connect to your Red Hat OpenShift Container Platform (Note: this name has changed to OpenShift Data Foundation in version 4) cluster
  • Install RHACM
  • Install jq

Deploy Red Hat OpenShift Data Foundation

Follow this blog, Deploying your storage backend using OpenShift Container Storage 4 to deploy Red Hat OpenShift Data Foundation.

Enable Red Hat Advanced Cluster Management Observabilty Service with Red Hat OpenShift Data Foundation

After the OpenShift Data Foundation is installed, complete the following steps to enable the RHACM observabilty service with Red Hat OpenShift Data Foundation:

  1. Log in to your RHACM hub cluster.

  2. Create a namespace for the observability service with the following command:

    oc create namespace open-cluster-management-observability

  3. Generate your pull-secret. If RHACM is installed in the open-cluster-management namespace, run the following command:

    DOCKER_CONFIG_JSON=`oc extract secret/multiclusterhub-operator-pull-secret -n open-cluster-management --to=-`

    If the multiclusterhub-operator-pull-secret is not defined in the namespace, copy the pull-secret from the openshift-config namespace into the open-cluster-management-observability namespace. Run the following command:

    DOCKER_CONFIG_JSON=`oc extract secret/pull-secret -n openshift-config --to=-`

    Then, create the pull-secret in the open-cluster-management-observability namespace, run the following command:

    oc create secret generic multiclusterhub-operator-pull-secret \
        -n open-cluster-management-observability \
        --from-literal=.dockerconfigjson="$DOCKER_CONFIG_JSON" \
        --type=kubernetes.io/dockerconfigjson

  4. Access the object storage configuration from your OpenShift Data Foundation service, and then create the secret for your object storage:

    • Run the following command to get access key:

      $ ACCESS_KEY=$(oc get secret noobaa-admin -n openshift-storage -o json | jq -r '.data.AWS_ACCESS_KEY_ID|@base64d')

      $ echo $ACCESS_KEY
      YOUR_ACCESS_KEY_VALUE

    • Run the following command to get secret access key:

      $ SECRET_KEY=$(oc get secret noobaa-admin -n openshift-storage -o json | jq -r '.data.AWS_SECRET_ACCESS_KEY|@base64d')

      $ echo $SECRET_KEY
      YOUR_SECRET_KEY_VALUE

    • Run the following command to get endpoint:

      oc get noobaa -n openshift-storage -o yaml

      ...

         serviceS3:
             externalDNS:
             - YOUR_END_POINT_URL_VALUE
             - YOUR_END_POINT_URL_VALUE
      ...

      Your secret might resemble the following file:

      apiVersion: v1
      kind: Secret
      metadata:
        name: thanos-object-storage
        namespace: open-cluster-management-observability
      type: Opaque
      stringData:
        thanos.yaml: |
          type: s3
          config:
            bucket: first.bucket
            endpoint: YOUR_END_POINT_URL_VALUE
            insecure: true
            access_key: YOUR_ACCESS_KEY_VALUE
            secret_key: YOUR_SECRET_KEY_VALUE

    • Run the following command to create the secret for your object storage:

      oc apply -f thanos-object-storage.yaml -n open-cluster-management-observability

  5. Create the MultiClusterObservability custom resource to enable the observability service. Your custom resource might resemble the following file:

    apiVersion: observability.open-cluster-management.io/v1beta2
    kind: MultiClusterObservability
    metadata:
      name: observability
      annotations:
    spec:
      observabilityAddonSpec:
        enableMetrics: true
        interval: 30
      storageConfig:
        metricObjectStorage:
          key: thanos.yaml
          name: thanos-object-storage

    • Run the following command to create your custom resource:

      oc apply -f multiclusterobservability_cr.yaml

    • Check the custom resource status. If you receive the following message, the observability service is enabled successfully: Observability components are deployed and running. Run the following command to view the status:

      oc get mco observability -o yaml

      ...
      status:
        conditions:
        - lastTransitionTime: "2021-06-10T07:57:23Z"
          message: Installation is in progress
          reason: Installing
          status: "True"
          type: Installing
        - lastTransitionTime: "2021-06-10T07:59:24Z"
          message: Observability components are deployed and running
          reason: Ready
          status: "True"
          type: Ready

Conclusion

Throughout the blog, I introduce a storage solution for Red Hat Advanced Cluster Management observabilty service. In addition, you can also use Amazon S3 (or other S3 compatible object stores like Ceph), Google Cloud Storage, or Azure storage as the storage service. For more details on Red Hat Advanced Cluster Management observabilty service, see the product documentation.

I hope that this article was useful to you. Please don’t hesitate to reach out if you have any feedback!

Categories

Storage, How-tos, Monitoring, Red Hat Advanced Cluster Management, Red Hat OpenShift Container Storage, Red Hat OpenShift Data Foundation

< Back to the blog