Creating and granting access to Amazon S3 bucket
There are actually several ways to go about granting S3 access to the pods running in your Red Hat® OpenShift® Service on AWS (ROSA) cluster. For example, you can set the credentials as environment variables in the notebook using pod identity/Identity Access Management (IAM) Roles for Service Accounts (IRSA) to authenticate the pods to S3. Or, you can install the AWS command line interface (CLI) in the cluster, among others.
For the sake of simplicity, we’ll install the CLI in the cluster and then use the command aws configure to provide the credentials. Be sure that you have your AWS access key and secret access key handy. You could create new keys in the IAM section from the AWS console if you have lost yours.
What will you learn?
- Creating the Amazon S3 bucket
- Granting access to the Amazon S3 bucket
What do you need before starting?
- Met all prerequisites
- Have AWS access key and secret access key
Steps to create the Amazon S3 bucket
- Now log into your cluster and go to the namespace where your notebook is located:
oc project rhods-notebooks - Run the following command and make sure that your pods are running:
oc get pods - Once you have the name of the pod (in this case it is called
jupyter-nb-admin-0), execute into it:oc exec -it jupyter-nb-admin-0 -- /bin/bash Next, let's install the AWS CLI in that pod:
curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip" unzip awscliv2.zip ./aws/install -i ~/.local/aws-cli -b ~/.local/bin- Then modify your PATH environment:
export PATH=~/.local/bin:$PATH - Once it is correctly installed, be sure that you have your AWS Access Key ID and AWS Secret Access Key ready, and run the following command:
aws configure - Select the region where your cluster is located. You could verify the configuration by running simple command such as listing the S3 buckets:
aws s3 ls - Once the credentials matter is sorted, let's create an S3 bucket in your AWS account. Again, there are many ways to go about this. The easiest would be to go to your AWS console and create the bucket in your region from there and leave all the settings to default. Alternatively, you can run this command to create a bucket (in this case, it’s called
llm-bucket-dsari, and the cluster region isus-west-2): aws s3 mb s3://llm-bucket-dsari --region us-west-2
Once these steps are complete, you’re ready to begin training the LLM model in the next resource.
