Using libguestfs Tools to Manipulate VM images in OpenShift Virtualization

The libguestfs tools make working with virtual machine disk images easy. Typically run from the command line of a Linux machine, this tool set is also available to be used with virtual machine disk images located on PVCs in OpenShift Virtualization as well through the virtctl client tool. In this demonstration, we'll use these tools to modify an image located on a PVC that could be used as a template to deploy multiple database servers.

The disk image used in this demonstration is a "QCOW2" created using image builder from Red Hat Insights. The image builder tool located at the link makes it simple to customize an image for virtually any purpose and environment, from ISOs to cloud images. You can add in any packages you may need and even customize the storage directly. For the sake of this article, the image is a minimal one with no additional packages or custom partitioning.

Below are the steps we'll take to customize the image to suit our needs using the libguestfs tools.

1. Copy a sample database file into the mounted image on the PVC.
2. Reset the root password.
3. Register with Red Hat Subscription Manager to enable installing and updating from Red Hat.
4. Install postgresql on the disk image.
5. Run all updates using the default package manager.
6. Run SE Linux relabel to correct any potential inconsistencies.
7. Deregister with subscription manager in order to not have duplicate systems in our subscription.

To accomplish the tasks listed above we need to have, at a minimum, edit permissions for the OpenShift project in which our PVC resides. We will be using a project created by the user "testuser" named beehive-virt. Note we are modifying a disk image on a template PVC from which virtual machines will be created, and not a PVC that holds a disk image for an existing virtual machine. The libguestfs tools, however, can be used for that purpose as well.

Uploading the unmodified image to be used

The first step we'll take is to upload the basic image of RHEL 9 to a PVC and, after it completes, we'll take a look at the image itself and begin the process of customizing. The first command we use will upload the initial image and create a PVC on which to store the data.

For the image upload, we use the image-upload option of the virtctl tool, which is available from the help menu (the "?" icon) in the OpenShift GUI. This command then uploads our image using the Containerized Data Importer (CDI) from the CLI. Depending on the size of the image being uploaded, we are able to upload the image in compressed form. The CDI then extracts the image during the upload process to the specified PVC. If uploading a compressed image, be sure that the PVC is at least 3% larger than the extracted image size to account for overhead requirements. The command and its output is below:

The insecure option is used here because there is no certificate installed in the test cluster.

virtctl image-upload -n beehive-virt pvc beehive-guest-image --size=15Gi --image-path=./beehive-guest-image.qcow2 --insecure

PVC beehive-virt/beehive-guest-image not found 
PersistentVolumeClaim beehive-virt/beehive-guest-image created
Waiting for PVC beehive-guest-image upload pod to be ready...
Pod now ready
Uploading data to https://cdi-uploadproxy-openshift-cnv.apps.beehive.example.org

 669.38 MiB / 669.38 MiB [==============================================================================================================================================] 100.00% 45s

Uploading data completed successfully, waiting for processing to complete, you can hit ctrl-c without interrupting the progress
Processing completed successfully
Uploading ./beehive-guest-image.qcow2 completed successfully

To show that the PVC has been created run:

oc get pvc -n beehive-virt

NAME                       STATUS     CAPACITY   ACCESS MODES 
beehive-guest-image        Bound      15Gi       RWX

Launching the libguestfs tools and gathering guest image information

In order to work on the disk image, we need to use the virtctl guestfs command along with the PVC on which the disk image resides. In our case it's beehive-guest-image. This command tells us that the image was automatically mounted at /dev/vda when we invoked the virtctl command; we will be customizing this mounted image. Below are the commands and their output:

virtctl guestfs beehive-guest-image

Use image: registry.redhat.io/container-native-virtualization/libguestfs-tools@sha256:26eb60f6fcdba7c90b82401c7e0b3f4e21b2b57c51d53e9a421f2fc7b2c8ec0e 
The PVC has been mounted at /dev/vda 
Waiting for container libguestfs still in pending, reason: ContainerCreating, message:  
If you don't see a command prompt, try pressing enter.+ touch /usr/local/lib/guestfs/appliance/done + /bin/bash
bash-4.4$

The shell prompt shows that the command was successful, so first we'll find out some information about the disk image. One way to do this is by running the virt-filesystem command. This will give us a basic idea of what size partitions and file systems are on the image. See the example below, and the output:

virt-filesystems -a /dev/vda -l -h

Name       Type        VFS   Label  Size  Parent
/dev/sda2  filesystem  vfat  -      200M  -
/dev/sda3  filesystem  xfs   boot   495M  -
/dev/sda4  filesystem  xfs   root   9.3G  -

We can also run virt-df which gives us different, but no less useful, information:

virt-df -a /dev/vda -h**

Filesystem                                Size       Used  Available  Use%
vda:/dev/sda2                             200M       7.0M       193M    4%
vda:/dev/sda3                             495M        79M       415M   17%
vda:/dev/sda4                             9.3G       1.1G       8.2G   12%

Customizing a Virtual Machine Disk Image Located on a PVC

While still working on the pod created previously, we will complete the tasks listed in the introduction. These commands can be run one at a time manually, strung together with all options, or by creating a file that will serve as a sort of "answers" file. We'll explore each option and its output.

For the purposes of this example, we have previously downloaded a sample database file using curl to /tmp in the pod. This command requires the pod to have network connectivity, which by default, it does. We now use virt-copy-in to copy that file into the mounted VM image and verify its success:

virt-copy-in -a /dev/vda /tmp/postgre_sample_data.sql /opt/

virt-ls -a /dev/vda /opt

postgre_sample_data.sql

Reset the root password:

virt-customize -a /dev/vda --root-password password:redhat1234

[   0.0] Examining the guest ...
[   5.5] Setting a random seed
[   5.6] Setting the machine ID in /etc/machine-id
[   5.6] Setting passwords
[   7.3] Finishing off

Add credentials with which to register with Red Hat subscription manager, then register the system.

virt-customize -a /dev/vda --sm-credentials<username>:password:<password> --sm-register

[   0.0] Examining the guest ...
[   5.3] Setting a random seed
[   5.4] Registering with subscription-manager
[  28.2] Finishing off

Install postgresql on the disk image:

virt-customize -a /dev/vda --install postgresql

[   0.0] Examining the guest ...
[   5.3] Setting a random seed
[   5.3] Installing packages: postgresql
[  13.0] Finishing off

Run all updates using the default package manager, which will vary depending on the operating system used on the image:

virt-customize -a /dev/vda --update

[   0.0] Examining the guest ...
[   5.3] Setting a random seed
[   5.3] Updating packages
[ 149.2] Finishing off

Run SELinux relabel in order to fix any file inconsistencies:

virt-customize -a /dev/vda --selinux-relabel

[   0.0] Examining the guest ...
[   4.8] Setting a random seed
[   4.8] SELinux relabelling
[  20.4] Finishing off

Deregister with subscription manager in order to not have duplicate systems in our subscription:

virt-customize -a /dev/vda --sm-unregister

[   0.0] Examining the guest ...
[   5.5] Setting a random seed
[   5.5] Unregistering with subscription-manager
[   6.6] Finishing off

The examples above illustrate the ability to run commands ad hoc, but we are also able to shorten the process by using a file that contains the commands we want to run. The file was created on the libguestfs pod itself using vim, but it could just as easily be downloaded from a valid address, host, or elsewhere.

The commands we've added to the file "customize-items" to be run are:

root-password password:redhat1234
sm-credentials <username>:password:<password>
sm-register
install postgresql
update
selinux-relabel
sm-unregister

We then perform the customization using the "--commands-from-file" option along with the filename:

virt-customize -a /dev/vda --commands-from-file customize-items

[   0.0] Examining the guest ...
[   5.7] Setting a random seed
[   5.7] Registering with subscription-manager
[  13.6] Installing packages: postgresql
[  15.2] Updating packages
[  16.8] Unregistering with subscription-manager
[  17.6] Setting passwords
[  19.0] SELinux relabelling
[  33.4] Finishing off

Finally, we're able to customize using the image using a single command with the options listed below:

virt-customize -a /dev/vda --root-password password:redhat1234 --sm-credentials your_username:password:your_password --sm-register --install postgresql --update --selinux-relabel --sm-unregister

[   0.0] Examining the guest ...
[   5.4] Setting a random seed
[   5.4] Registering with subscription-manager
[  19.8] Installing packages: postgresql
[  22.9] Updating packages
[  24.5] Unregistering with subscription-manager
[  25.3] Setting passwords
[  27.1] SELinux relabelling
[  43.4] Finishing off

The examples above are just a few ways in which a disk image can be modified using the libguestfs tools on a PVC. Scenarios where these tools would be useful could be ones where large disk images would prevent a local download in order to edit, or where security policies would prevent virtual machine administrators or users from holding disk images on local storage. For whatever reasons that are chosen, the libguestfs tools are a great way to maintain and modify virtual machines without having to create one and go through all the needed steps for post creation customizing. Much more can be done with them, and you can find out more at the main page of the tools at libguestfs.org.