This is part 2 of a 5 part series. Links to the other parts are here:

• Introduction to OpenShift Pipelines
• Using Source 2 Image Build in Tekton
• Manage a Runtime Image
• Application Deployment and Pipeline Orchestration
• Using the Examples in this Series

Introduction

OpenShift Pipelines is a Continuous Integration / Continuous Delivery (CI/CD) solution based on the open source Tekton project. The first article in this series introduced the main concepts of Tekton and the resources that are used to construct a pipeline. In this article the Red Hat OpenShift source 2 image process is examined in detail, and Tekton tasks are used to take advantage of source 2 image for software build and container creation.

Access to the source content

All assets required to create your own instance of the resources described in this article can be found in the GitHub repository here. In an attempt to save space and make the article more readable only the important aspects of Tekton resources are included within the text. Readers who want to see the context of a section of YAML should clone or download the Git repository and refer to the full version of the appropriate file.

Source to Image Build in a Tekton pipeline

Background on the Source to Image process

The Red Hat OpenShift ‘Source to Image’ (S2I) build process is a fantastic capability that allows a developer to point OpenShift at a Git source repository and OpenShift will perform the following tasks :

1. Examine the source code in the repository and identify the language used
2. Select a builder image for the identified language from the OpenShift image repository
3. Create an instance of the builder image from the image repository (green arrow on figure 1)
4. Clone the source code in the builder image and build the application (the grey box in figure 1). The entire build process including pulling in any dependencies takes place within the builder image.
5. When the build is complete push the builder image to the OpenShift image repository (the blue arrow on figure 1).
6. Create an instance of the builder image, with the built application, and execute the container in a pod (the purple arrow in figure 1).

Figure 1 - Source to image process

All of the above actions take place simply by selecting a Git repository as shown in figure 2.

Figure 2 - Selecting a Git repository from which the source code should be built

Note that in figure 2 the Node.js language has been identified in the source code, and figure 2 also shows the full range of languages that are supported by S2I (as of OpenShift 4.5).

After a few seconds OpenShift created a number of resources as shown in figure 3, which are :

1. A build configuration to perform the build
2. A build has executed resulting in a new container image being created
3. The container has been started within a pod
4. A service has been created to communicate with the container in the pod
5. A route has been created to communicate with the service

All of the above has been created simply by pointing OpenShift at a Git repository.

Figure 3 - The result of a source to image build process

Creating a runtime image without source code

The source to image process is fantastic for creating running applications from source code quickly to accelerate development processes. However some customers want to have more control over the creation of a runtime image that is to be used in production. For example some customers don’t want the builder tools and source code in a production image, while others want to be able to create a master image into which the binary application should be injected before the image progresses through testing and pre-production environments on the way to production.

It is still possible to take advantage of the capabilities of the source to image process by treating the resulting image as an intermediate image and then extracting the binary executable and pushing it into a second runtime image. This process will be explained in the following section.

The summary of the steps performed in this stage are:

1. Create a configuration file to provide a mechanism for modifying the characteristics of the Maven build
2. Use the Source to Image process to create a dockerfile that performs the build
3. Use the Buildah image creation utility to execute the generated dockerfile which will perform the build of the example java application
4. Create a second dockerfile that will extract the deliverable and put it into the runtime image
5. Use the Buildah image creation utility to execute the generated dockerfile which will produce the runtime image containing the executable
6. Push the image to the OpenShift registry

Source to Image Build process

The Tekton task for this stage of the process is in the file /build/tasks/build.yaml

The first stage of the source to image process is to create an environment file with name and value properties such as:

MAVEN_CLEAR_REPO=<value>
MAVEN_ARGS_APPEND=<value>
MAVEN_MIRROR_URL=<value>

Values are supplied as parameters to the task, from the pipeline, or they can be set as default values in the task file.

The second stage of the build task is to use the source to image ‘build’ process to create a dockerfile. The step within the task is shown below, with some parameters are resources that require explanation.

spec:
 params:
   - name: PATH_CONTEXT
     description: Path of the source code
     type: string
     default: .
 resources:
   inputs:
     - name: source
       type: git
 steps:
   - name: generate
     command:
       - s2i
       - build
       - $(params.PATH_CONTEXT)
       - registry.access.redhat.com/redhat-openjdk-18/openjdk18-openshift
       - '--image-scripts-url'
       - 'image:///usr/local/s2i'
       - '--as-dockerfile'
       - /gen-source/Dockerfile.gen
       - '--environment-file'
       - /env-params/env-file
     image: registry.redhat.io/ocp-...-preview/source-to-image-rhel8
     volumeMounts:
       - name: gen-source
         mountPath: /gen-source
       - name: envparams
         mountPath: /env-params
     workingDir: /workspace/source

The reference to the Git resource at the top of the task file has a name of ‘source’. This ensures that the Git repository will be cloned to a directory of /workspace/source as shown in figure 4 labelled 1. The working directory of the step is set to this location and the s2i build process expects to find the source code in the location, identified by the parameter $(params.PATH_CONTEXT), which is set to the value of ‘.’ as a default value.

Figure 4 - File movement in source to image build phase

The source to image build process refers to an openjdk18-openshift image that is used as the builder image and identifies where the source to image binaries are to be found in that image. The next argument identifies where the dockerfile is to be located and the file name as indicated by label 2 in figure 4. This location is very important because in addition to putting the dockerfile in the location the source to image build process also copies the source code from the source code location (current working directory) to the path :

<location-of-dockerfile>/upload/src.

This process is indicated by stage 3 of figure 4.

In this example the current working directory and the path context supplied to the s2i build command are the same thing. If the path context is given as a subdirectory of the current working directory then it is this location that will be copied to the <location-of-dockerfile>/upload/src location.

Consuming the dockerfile - Running the build

The next stage of the process is to use the buildah command to execute the dockerfile created in the last step. The yaml for this step is shown below:

 - name: build
     command:
       - buildah
       - bud
       - '--tls-verify=$(params.TLSVERIFY)'
       - '--layers'
       - '-f'
       - /gen-source/Dockerfile.gen
       - '-t'
       - $(resources.outputs.intermediate-image.url)
       - .
     image: registry.redhat.io/rhel8/buildah
     resources: {}
     securityContext:
       privileged: true
     volumeMounts:
       - name: pipeline-cache
         mountPath: /var/lib/containers
       - name: gen-source
         mountPath: /gen-source
     workingDir: /gen-source

When the dockerfile is executed it will begin by copying the source code from the location <current-directory>/upload/src to the location /tmp/src within the container image. All build activity will then take place within the image at this local directory. Source code is expected to be in a location of /tmp/src/src and any deliverables will be produced in the directory /tmp/src/target as shown by figure 5.

Figure 5 - source to image docker file content locations

Two volume mounts are used in this example. The first is called gen-source which is where the dockerfile was created and the second is from a persistent volume claim which is pipeline-cache. The second volume has been mounted to /var/lib/containers since this is the location where Buildah expects to find the local image repository.

The result of running this step will be a new image created by the Buildah command in the local Buildah repository under the identifier of the URL for the intermediate-image resource. The intermediate image resource definition is shown in the images section above.

Summary

This article showed the use of Tekton tasks to perform the OpenShift source 2 image build process, enabling the user to take advantage of source 2 image as part of a wider software build / test / deploy process. This article also showed the file movements that take place during a source 2 image build execution.

What’s next

The next article in this series covers image management using Tekton and will show how a runtime image can be created by extracting the built objects from the builder image created in this article. It will also show how to push the image to OpenShift (ready for deployment and test) and to quay.io (ready for download and use by other clusters such as production).