Using OpenShift Lightspeed with ARO

This guide walks through setting up OpenShift Lightspeed backed by Azure AI Foundry services for the LLM.

Prerequisites

• An ARO Cluster is already installed.
• Permissions to use/register Microsoft Cognitive Services

Command line tools used in this guide:

• aws cli
• jq

Set up environment

Create environment variables.

Install the necessary operator, OpenShift Lightspeed Operator from OpenShift console.

Prepare the LLM for serving OpenShift Lightspeed

1. Create the Azure Cognitive Service.

2. Create the deployment. In this case we will use the model gpt-4o-mini.

3. Validate the deployment exists.

4. Retrieve the endpoint’s url and api-token.

5. We must use an app registration to get access to the AI service.

6. Prepare the data, since this must be encoded base 64.

OpenShift LightSpeed Configuration

1. Create the secret with the credentials and the OLSConfig.

2. We should wait a couple of minutes until the lightspeed pod server is running.

Test.

To test, try querying the OpenShift Lightspeed in ARO.

Cleanup

1. Delete OpenShift configurations

2. Uninstall the OpenShift Lightspeed Operator from the Operator Hub in the OpenShift console.

3. Delete the cognitive services account and the service principal