Managing worker nodes in Red Hat OpenShift Service on AWS clusters
When using your cluster, there may be times when you need to change aspects of your worker nodes like scaling, changing the type, adding labels or taints, to name a few.
Most of these things are done through the use of machine pools in Red Hat® OpenShift® Service on AWS (ROSA). Think of a machine pool as a “template” for the kinds of machines that make up the worker nodes of your cluster. A machine pool allows users to manage many machines as a single entity. Every ROSA cluster has a "Default" machine pool created when the cluster is created. If you'd like to learn more see About machine pools and autoscaling.
What will you learn?
- How to create a machine pool using the rosa command line interface (CLI)
- How to create a machine pool using the OpenShift Cluster Management (OCM) user interface (UI)
- How to scale worker nodes
- How to add node labels
- How to mix different node types
Creating a machine pool using the rosa CLI
rosa create machinepool --cluster=<cluster-name> --name=<machinepool-name> --replicas=<number-nodes>
$ rosa create machinepool --cluster=my-rosa-cluster --name=new-mp --replicas=2 I: Machine pool 'new-mp' created successfully on cluster 'my-rosa-cluster' I: To view all machine pools, run 'rosa list machinepools -c my-rosa-cluster'
- Sometimes it is beneficial to add node label(s) and/or taints. One use case is to target certain workloads to specific nodes. Let’s say we want to run our database on specific nodes. We can add node labels to the worker nodes when we create a new machine pool using the CLI.
rosa create machinepool --cluster=<cluster-name> --name=<machinepool-name> --replicas=<number-nodes> --labels=’<key=pair>’
$ rosa create machinepool --cluster=my-rosa-cluster --name=db-nodes-mp --replicas=2 --labels='app=db','tier=backend' I: Machine pool 'db-nodes-mp' created successfully on cluster 'my-rosa-cluster'
- This will create an additional 2 nodes that you can manage as one unit and also assign them the labels shown.
- Now run the following to see the new machine pool created along with the labels we gave.
rosa list machinepools --cluster=<cluster-name>
ID AUTOSCALING REPLICAS INSTANCE TYPE LABELS TAINTS AVAILABILITY ZONES Default No 2 m5.xlarge us-east-1a new-mp No 2 m5.xlarge us-east-1a db-nodes-mp No 2 m5.xlarge app=db, tier=backend us-east-1a
Creating a machine pool with the OCM UI
- In the OCM user interface, click on your cluster.
- Then click on the "Machine Pools" tab.
- Click the "Add Machine pool" button.
- Fill in the desired configuration. At this point you can also expand the "Edit node labels and taints" section to add node labels and taints to the nodes in this machine pool.
- You will now see the new machine pool you created in the UI.
Scaling worker nodes
- To scale the number of worker nodes, we need to edit the machine pool they belong to. The default machine pool is called “Default” which is created with every cluster. We could also use the one we created before.
- We should see the “Default” pool that is created with each cluster.
rosa list machinepools --cluster=<cluster-name>
ID AUTOSCALING REPLICAS INSTANCE TYPE LABELS TAINTS AVAILABILITY ZONES Default No 2 m5.xlarge us-east-1a
- To scale this out to 3 nodes run
rosa edit machinepool --cluster=<cluster-name> --replicas=<number-nodes> <machinepool-name>
rosa edit machinepool --cluster=my-rosa-cluster --replicas 3 Default
- Run the following to see that it has taken effect:
rosa describe cluster --cluster=<cluster-name> | grep Compute
You will see a response showing 3 compute nodes:
$ rosa describe cluster --cluster=my-rosa-cluster | grep Compute - Compute: 3 (m5.xlarge)
- (Optional) One can also scale the cluster from the OCM UI by clicking on the "three dots" on the right of the machine pool you want to edit and clicking "scale".
- We can also confirm this by accessing OCM and selecting the cluster.
- On the overview tab, scroll down to the middle section under details you will see Compute listing "3/3".
Adding node labels
NOTE: Labels or taints cannot be added to the "Default" machine pool (yet).
- Adding node label(s) can be achieved by the following command
rosa edit machinepool --cluster=<cluster-name> --replicas=<number-nodes> --labels='key=value' <machinepool-name>
For example if we wanted to add 2 labels to the new machine pool we created:
rosa edit machinepool --cluster=my-rosa-cluster --replicas=2 --labels 'foo=bar','baz=one' new-mp
- This command replaces all machine pool configurations with what is defined. If you just want to add another label and keep the old, you must state all the labels, otherwise it will replace anything existing with the one you had wanted to add. Similarly, if you want to delete a label only state the ones you want, excluding the one you want to delete.
Mixing different node types
- You can also mix different worker node machine types in the same cluster by using new machine pools. You cannot change the node type of a machine pool once created, but we can create a new machine pool with different nodes by adding the
- If we take the use case above (database nodes) but instead wanted to have a different node type when creating it, we would have run:
rosa create machinepool --cluster=<cluster-name> --name=<mp-name> --replicas=<number-nodes> --labels=’<key-pair>’ --instance-type=<type>
rosa create machinepool --cluster=my-rosa-cluster --name=db-nodes-large-mp --replicas=2 --labels='app=db','tier=backend' --instance-type=m5.2xlarge
- If you’d like to see all the instance types available you can run:
rosa list instance-types
- Or to make the decisions step-by-step, then use the
rosa create machinepool -c <cluster-name> --interactive
- List the machine pools to see the new larger instance type.
rosa list machinepools -c <cluster-name>
You are now ready to set up cluster autoscaling.