How to run and deploy LLMs using Red Hat OpenShift AI on a Red Hat OpenShift Service on AWS cluster

Learn how to install the Red Hat® OpenShift® AI (RHOAI) operator and Jupyter notebook, create an Amazon S3 bucket, and run the LLM model on a Red Hat OpenShift Service on AWS (ROSA) cluster.

Disclaimer: this content is authored by Red Hat experts, but has not yet been tested on every supported configuration.

Learn how to install the Red Hat® OpenShift® AI (RHOAI) operator and Jupyter notebook, create an Amazon S3 bucket, and run the LLM model on a Red Hat OpenShift Service on AWS (ROSA) cluster.

Disclaimer: this content is authored by Red Hat experts, but has not yet been tested on every supported configuration.

Next steps after running and deploying an LLM using Red Hat OpenShift AI

2 mins

Congratulations! You have deployed and trained an LLM using Red Hat® OpenShift® AI (RHOAI) on a Red Hat OpenShift Service on AWS (ROSA) cluster.

After completing this tutorial, you now have experience:

  • Installing RHOAI and Jupyter notebook
  • Creating and granting access to S3 bucket
  • Training LLM model
  • Future research
  • Performing hyperparameter tuning 

What comes next?

Next, watch a demonstration of a typical RHOAI workflow that includes text-to-image generation, creating a project, launching a Jupyter notebook with appropriate cluster resources, and training a foundation model from Hugging Face with one’s own data. Once the model is fine-tuned, the demonstrator also automates the build using a data science pipeline and serves the model for use in an AI-enabled application. 

Previous resource
Hyperparameter tuning

This learning path is for operations teams or system administrators.

Developers might want to check out how to create a natural language processing (NLP) application using Red Hat OpenShift AI on developers.redhat.com.

Get started on developers.redhat.com

Hybrid Cloud Logo LinkedIn YouTube Facebook Twitter

Products

Tools

Try, buy, sell

Communicate

About Red Hat

We’re the world’s leading provider of enterprise open source solutions—including Linux, cloud, container, and Kubernetes. We deliver hardened solutions that make it easier for enterprises to work across platforms and environments, from the core datacenter to the network edge.