Using MetalLB to access OpenShift Virtual Machines

In our previous article regarding high availability, we configured a Galera database cluster within MariaDB on three OpenShift VMs. In this article we are going to expand on that and grant access to the database from outside of the cluster using the MetalLB operator that is available for OpenShift.

Installation and Initial Configuration of the MetalLB Operator

First we'll need to install the MetalLB operator, and configure three items once that is finished. Installation of the MetalLB operator is simple enough, and can be done from the OperatorHub in the OpenShift UI by a user with privileges that are sufficient to do so. This is done from the OperatorHub in OpenShift, and we've installed it here using the default options.

After the MetalLB operator is installed we then need to create our first instance using the "Create MetalLB" button in the MetalLB tab of the operator. Since we're only going to have a single MetalLB instance, we can accept the minimal defaults. From there click the create button and the first step is complete. Below is where you'll find the button.

Next we allocate our IP addresses that MetalLB will use to assign an external IP address to whatever resource with which you are working. In our case, it will be a service that we will configure to allow entry into our Galera cluster. To do this we click the IPAddressPool tab in the same area as our MetalLB instance. Click the create button and it will pull up the yaml editor. Any or all of the items listed in the 'addresses' spec can be used.

kind: IPAddressPool
apiVersion: metallb.io/v1beta1
metadata:
  name: ip-addresspool-beehive
  namespace: metallb-system
spec:
  addresses:
    - <IPv4 first address> - <IPv4 last address>

For simplicity's sake we are using the metallb-system namespace. The name of the pool can be whatever is applicable to your environment. Under "addresses", enter the range to be used in CIDR notation or a start - end range. IPv6 is also supported, and both can be used simultaneously, but for our example we will be using IPv4. Multiple ranges are also supported, but for the purpose of this blog we have configured a single range. After we've clicked create, our second item is complete.

Lastly, in order to get traffic to our service, we are going to create a L2Advertisement. This is what MetalLB will listen on to route requests and traffic to our cluster via the service we plan to create. In keeping this as simple as possible, we are going to create an empty L2Adverstisement. This means that MetalLB will advertise all the services to our address pool we defined previously. This can be tailored to the needs of the organization to use only predefined address pools.

Under L2Advertisement click Create L2Advertisement. In the yaml editor that pops up, you can specify the name, namespace, and address pool. To create the aforementioned empty advertisement, remove the lines below the namespace definition. Click create when ready and our initial preparation steps are complete. See below for the yaml used in the l2advertisement.

apiVersion: metallb.io/v1beta1
kind: L2Advertisement
metadata:
  name: empty *This can be whatever you like*
  namespace: metallb-system

Service creation and IP assignment

Now that MetalLB is configured, let's use it to get traffic to our Galera cluster. We are going to create a service that gets assigned an IP address by MetalLB, then we will apply a label to our database servers that will be used in the service as a pod selector. For all intents and purposes, MetalLB is providing us a floating IP or VIP to ensure uptime on our database cluster.

First let's get the names of the virtual machines that we're planning to label. This can be accomplished by using the oc get vm command from the cli.

$ oc get vm

NAME              AGE   STATUS    READY
cnv-galera-db01   90d   Running   True
cnv-galera-db02   90d   Running   True
cnv-galera-db03   90d   Running   True

Now that we know the names of the vms that will be used in the MetalLB service are, they are labelled with the selector of our choosing. In this case it's 'app=galera-db-servers'. We can apply this label directly in the yaml of the OpenShift vms by using the 'oc edit' command or the 'oc patch' command. Whichever method is chosen, the vms will need to be restarted through the OpenShift UI or the 'oc' command and not simply an operating system reboot. This is so that the pod gets recreated and our label is properly applied. Below is the command and its output.

$ oc patch vm cnv-galera-db01 --type json -p '[{"op": "add", "path": "/spec/template/metadata/labels/app", "value": "galera-db-servers" }]'

virtualmachine.kubevirt.io/cnv-galera-db01 patched

Note that this needs to be done for all three of the database servers in the cluster that will be associated with the service we are creating.

Moving forward from here we will create a service with type LoadBalancer, which MetalLB will use to assign an IP address from the range we created. This will allow us to use that IP as the virtual IP for our database connections. To do that, we create a service much in the same way we do with any other OpenShift service. In this example we create a yaml file with the specifications we wish to apply. However, this could be copied and pasted into the UI or written out on the cli in a single command.

apiVersion: v1
kind: Service
metadata:
  name: galera-cluster-dbs
  namespace: <your namespace>
  annotations:
    MetalLB.universe.tf/address-pool: ip-addresspool-beehive *We have chosen this annotation so that MetalLB assigns an ip address from this pool specifically.*
spec:
  selector:
    app: galera-db-servers *This is the selector we created and applied in the above step.*
  ports:
    - port: 3306
      protocol: TCP
  type: LoadBalancer

After creating the service, MetalLB will then assign an ip address from the pool we've chosen, and will allow traffic to flow into and of out those vms. To show the route's status, we use the 'oc get' command.

$ oc get service galera-cluster-dbs -o wide

NAME                 TYPE           CLUSTER-IP      EXTERNAL-IP   PORT(S)          AGE   SELECTOR
galera-cluster-dbs   LoadBalancer   172.30.134.39   <assigned-ip> 3306:30763/TCP   10d   app=galera-db-servers

Finally, we test connectivity. There are different ways to do this, but below is a screenshot of the test used. The MySQL Workbench query was run on a physical workstation located outside of the cluster over a vpn connection, and the mysql cli query was run from the cnv-galera-db01 server.

Reliability in a few short steps

Having the ability to keep data intact and flowing is an essential part of a data driven world. Using the tools that are readily available within OpenShift Virtualization to make this process as simple as possible can lead to redundancy and data integrity that is as easily achieved as in a bare metal environment.

To read more about the MetalLB operator and its implementation, visit Federico Paolinelli's excellent blog post here.