Introduction

As Red Hat Advanced Cluster Management is written in Node JS for front-end and Go for back-end, the Red Hat Advanced Cluster Management development team (of course) started to find reusable methods through the different operator projects. I started a new project library-go where I added all helper methods. This project is Advanced Cluster Management agnostic and so can be reused by any Kubernetes developer.

The library-go can currently help you in the following areas described in the following section:

  1. TemplateProcessor and Applier packages: These two packages allow you to create/update/delete Kubernetes resources by providing templated .yaml files, string or bindata and a set of values. It is similar to Helm, but more lightweight.

  2. Client package: That package is meant to help you to create several types of Kubernetes clients.

  3. Webhook helper: This last package will create a service for the webhook server.

Find the code and packages

The code is located in an open repository open-cluster-management-library-go, allowing the community to contribute.

Packages in the repo

The repo contains multiple packages and is still growing. At publication of this blog, you can complete the following tasks with the repo:

  • Apply YAML in Kubernetes implemented in the ‘applier’ package. To learn how, see our Template document.

  • Create different Kubernetes clients, such as clientset, api, dynamic, and more. These clients are based on a kubeconfig file. that is implemented in the ‘client’ and ‘config’ package. Learn more about clients, see our Client.

  • Use Webhook helpers.

TemplateProcessor and Applier packages

You probably frequently use the CLI tools for Openshift (oc) or Kubernetes (kubectl) clients to apply a set of resources in a Kubernetes cluster with the oc apply -f command. The idea of the applier is to do the same in ‘Go’, but here you provide to the applier: a path where the resources templates. The applier will read them, sort them to avoid dependencies issues, and finally create or update these resources on the Kubernetes cluster.

The applier is divided into 2 main functionalities:

  • Processing of resource templates

  • Creation or deletion of these resources into the Kubernetes cluster

The template processing is done by calling the NewTemplateProcessor method, which will create a TemplateProcessor structure. See the following example:

//TemplateProcessor this structure holds all objects for the TemplateProcessor
type TemplateProcessor struct {
//reader a TemplateReader to read the data source
reader TemplateReader
//Options to configure the TemplateProcessor
options *Options
}

Next, the creation of the resource is done by calling NewApplier(), which will create Applier structure. See the following example:

//Applier structure to access Kubernetes through the applier
type Applier struct {
//An TemplateProcessor
templateProcessor *TemplateProcessor
//The client-go kubernetes client
client client.Client
//The owner of the created object
owner metav1.Object
//The scheme
scheme *runtime.Scheme
//A merger defining how two objects must be merged
merger Merger
//applier options for the applier
applierOptions *Options

}

The applierOptions contains the client-go options, a dry-run, a back-off, and a force-delete option.

TemplateProcessor

The TemplateProcessor reads resource templates and renders them with the provided values. It uses the behind-the-scene standard Go “text/template”. In the template, you can use the functions defined in sprig or add new functions by extending the templatefunction.go file.

To create a TemplateProcessor, you will need to call the NewTemplateProcess function and give a TemplateReader and Options. TemplateProcessor defines an interface to read resources from different inputs, the interface is named TemplateReader.

TemplateReader

The TemplateReader is an interface that defines functions to retrieve an asset template the list of assets template and to transform an asset template to JSON.

I developed an implementation of the TemplateReader interface to read files or a string. As a result, you can easily create an implementation for bindata, as you will see in later examples. Additionally, see the documentation at go-bindata.

See the TemplateReader interface in the following code:

//TemplateReader defines the needed functions
type TemplateReader interface {
//Retrieve an asset from the data source
Asset(templatePath string) ([]byte, error)
//List all available assets in the data source
AssetNames() ([]string, error)
//Transform the assets into a JSON. This is used to transform the asset into an unstructrued.Unstructured object.
//For example, if the asset is a YAML, you can use yaml.YAMLToJSON(b []byte) as implementation as it is shown in
//testread_test.go
ToJSON(b []byte) ([]byte, error)
}

Also, see the following different implementations of a TemplateReader interface:

Assets from a directory can be found at directory assets reader.

Assets from bindata can be found at bindata assets read.

Assets from a string can be found at string assets reader.

Options

Currently we have three options. Two option allow you to overwrite the order in which the resources are sorted before they are created or deleted in Kubernetes. The third option is to specify which one of the two we want to use.

The default orders can be found in default resources order.

//defaultKindsOrder the default order
var defaultKindsOrder = []string{
"CustomResourceDefinition",
"ClusterRole",
"ClusterRoleBinding",
"Namespace",
"Secret",
"ServiceAccount",
"Role",
"RoleBinding",
"ConfigMap",
"Deployment",
}

Methods

There are mainly two types of methods for the TemplateProcessor.

  • The first methods return a sorted list of unstructured.Unstructured objects
  • The second methods return an array of strings or an array of arrays of bytes.

Additionally, some helper methods are available to transform array of arrays of bytes to an array of references of unstructured.Unstructured.

The input for these methods are either a path toward multiple assets, or a single asset path and a structure of values. If the input is a path, you can also mention a list excluded asset/path.

In these methods, you can define whether you want to find the assets recursively in the path.

See the following example of TemplateProcessor usage:

  1. Example based on a path and providing a sorted array of objects

  2. Example based on a list of assets

Next, use the Applier to apply your YAML resources.

Applier

The Applier uses the TemplateReader to create, update, or delete Kubernetes resources from an array of unstructured.Unstructured objects.

To create an Applier, you must call the NewApplier method and provide the TemplateReader, TemplateProcessor Options, a Kubernetes client, the owner of the resources that will be created, the scheme, and a merger.

If the owner and the scheme are provided, then all resources are created with an ownerReference toward that owner. A default merger exists and is called DefaultKubernetesMerger. This method will overwrite the existing resource root attributes with the new one if it changed. The merger is used when you want to update a resource, rather than just create that resource. You can write your own merger, you just need to implement the applier.Merger interface.

The applier uses the backoff functionality, meaning if a create, update, or delete failed, the backoff functionality retries a number of times before raising the error.

Like the processor, the Applier also has a number of methods. I illustrate by example the following main methods:

The first method, I use a path to create the resources. See the following example based on a path.

The second method, I use a list of .yaml files to create resources. See the following example based on different .yaml file locations.

Client package

The client package allows you to quickly create Kubernetes clients based on the location of the kubeconfig, the host URL, and the context.

You can create the following types of clients:

  • Controller-runtime client
  • Client-go clientset client
  • Client-go dynamic client
  • Apiextension-apiserver clientset client

If the kubeconfig parameter is not provided, then the method LoadConfig will check if the environment variable KUBECONFIG is set.

If the environment variable is set, it will use that environment variable. Otherwise, if the method runs in a cluster, the method will use the kubeconfig available in that cluster. Finally, if the method is not running in a cluster, then the method will look in your home directory to find the kubeconfig file.

If a Kubernetes kubeconfig file is provided, and if the Kubernetes context is provided, the method will use that context to determine the configuration to load. If the config file and URL is provided, but the context is not provided, the method will use that URL to determine the configuration to load. Now if the host URL is also not provided, the method will use the current context to determine the configuration.

The client package has also methods to check if a cluster has specified CRDs or Deployments.

Webhook helper

This package contains the WebHookWireUp Object, which provides a convenient way to wire up a webhook with ValidatingWebhookConfiguration and Service to the manager of your operator.

//WireUp defines the con
type WireUp struct {
mgr manager.Manager
stop <-chan struct{}

Server *webhook.Server
Handler webhook.AdmissionHandler
CertDir string
logr.Logger

WebhookName string
WebHookPort int

WebHookeSvcKey types.NamespacedName
WebHookServicePort int

ValidtorPath string

DeployLabel string
DeploymentSelector map[string]string
}

The service for webhook will be bound to the deployment of the webhook pod and map the ports; expose port 443 for the webhook service, which has container port as 9443 by defaul).

In addition, the Webhook package also provides helpers (certificate.go) to help with the following:

  1. Create a key pair at CertDir
  2. Create CA cert, which will be injected to the ValidatingWebhookConfiguration

The previous parameters can be modified passing the optional parameter to the constructor function NewWireUp. See the following example:

	wbhCertDir := func(w *WireUp) {
w.CertDir = filepath.Join(os.TempDir(), "k8s-webhook-server", "serving-certs")
}

wiredWebhook, err := NewWireUp(mgr, sig, wbhCertDir, wbhLogger, wbhCertDir)

The webhook helper will also create a service for the webhook server.

The service will bound to the deployment of the webhook pod and map the ports; expose port 443 for the webhook service, which has container port as 9443 by default.

		WebHookPort:        9443,
WebHookServicePort: 443,
...

func newWebhookServiceTemplate(svcKey types.NamespacedName, webHookPort, webHookServicePort int, deploymentSelector map[string]string) *corev1.Service {
return &corev1.Service{
ObjectMeta: metav1.ObjectMeta{
Name: svcKey.Name,
Namespace: svcKey.Namespace,
},
Spec: corev1.ServiceSpec{
Ports: []corev1.ServicePort{
{
Port: int32(webHookServicePort),
TargetPort: intstr.FromInt(webHookPort),
},
},
Selector: deploymentSelector,
},
}
}

Acknowledgment:

I would like to thank Ian Zhang for contributing to the library-go by adding the webhook helper.