Cloud Experts Documentation

Data science pipelines with Azure Blob storage on ARO

This content is authored by Red Hat experts, but has not yet been tested on every supported configuration.

OpenShift AI pipelines store their artifacts in S3-compatible object storage. Azure Blob storage doesn’t speak the S3 API. This guide puts the OpenShift Data Foundation Multicloud Object Gateway (MCG) in front of a Blob container, so pipelines get an S3 endpoint while the data stays in your Azure storage account.

MCG runs in standalone mode, so you don’t need the extra storage nodes that a full ODF deployment uses.

Prerequisites

  • OpenShift AI installed on ARO. See Installing Red Hat OpenShift AI 3.x on ARO .

  • The az CLI, logged in to the subscription that contains the cluster.

  • oc 4.18 or later (for oc wait --for=create).

  • ODF must not already be installed. If it is, skip Deploy the Multicloud Object Gateway and create only the backing store and bucket class in that section.

  • Set these variables:

Create the Blob container

Note: For simplicity, this guide authenticates with a storage account key. For production, restrict network access to the storage account (for example with a private endpoint in the cluster VNet), and plan for key rotation. Rotating the key means updating the azure-blob-creds secret.

Deploy the Multicloud Object Gateway

  1. Install the ODF Operator. The channel matches the cluster’s OpenShift minor version:

  2. Create a standalone MCG StorageCluster:

  3. Create a backing store for the Blob container and a bucket class that uses it:

Create a bucket for a data science project

  1. Create the project and make it visible in the dashboard:

  2. Request a bucket. MCG creates a ConfigMap and a Secret with the same name, holding the bucket details and the S3 credentials:

Configure the pipeline server

Point the pipeline server at the MCG service. The cluster service CA signs the service certificate. OpenShift publishes that CA to every namespace in the openshift-service-ca.crt ConfigMap:

Create the pipeline server:

Wait for the pipeline server to become ready:

Validate

  1. In the dashboard, open the project, go to Pipelines, and import a sample pipeline. Then create a run.

  2. When the run finishes, open one of its output artifacts. The dashboard reads it back through the S3 endpoint, which confirms both the write and the read.

  3. Check that MCG has written objects to the Azure container:

    MCG stores data in its own chunked format, so the blob names won’t match the pipeline artifact names.

Clean up

Warning: These steps permanently delete pipeline data. Run only the steps for resources you created with this guide.

Remove the project and its bucket:

Remove the shared storage only if no other project uses it. To check, run oc get obc -A.

Back to top

Interested in contributing to these docs?

Collaboration drives progress. Help improve our documentation The Red Hat Way.

Red Hat logo LinkedIn YouTube Facebook Twitter

Products

Tools

Try, buy & sell

Communicate

About Red Hat

We’re the world’s leading provider of enterprise open source solutions—including Linux, cloud, container, and Kubernetes. We deliver hardened solutions that make it easier for enterprises to work across platforms and environments, from the core datacenter to the network edge.

Subscribe to our newsletter, Red Hat Shares

Sign up now
© 2026 Red Hat