Have you ever found yourself forgetting to set some key attributes on your Kubernetes cluster? Or perhaps others have failed to set them? Maybe you have encountered annoying missing labels, incorrect security policies, or network settings.
Gatekeeper is a great tool for alleviating all this. Until recently, it allowed us to validate incoming resources and reject nonconformant ones based on user-defined policies. This is great but still left the burden of updating the faulty resources manually. With the new mutation feature, updating nonconformant resources can be automated.
How Gatekeeper Mutation Works
The gatekeeper mutation system introduces two new mutation policy CRD’s: Assign and AssignMetadata, which allow the user to define the desired mutations. When a mutation policy is created, the mutation system parses it and adds to a set of mutators. Whenever a Kubernetes resource is created, it is intercepted by the gatekeeper mutating admission webhook, the mutators are applied, and the resource is mutated
Defining Mutations
The mutation policies are defined by means of mutation specific CRDs:
- AssignMetadata - defines changes to the metadata section of a resource. The metadata section is sensitive to change, and hence has more limitations than other rules. Only labels and annotations can be modified
- Assign - any change outside the metadata section
Each mutation CRD consists of three major sections:
- Mutation scope - describes which resources are to be mutated
- Mutation intent - the location of the mutation and the value to be assigned
- Conditional - conditionals for performing the mutation
Mutation Scope
The mutation scope defines what resources will be mutated. It consists of two sections:
- applyTo: This defines the resources to be mutated by gvk. This allows gatekeeper to validate that the schema used in other parameters is correct (for example, the path to be modified). This is currently only valid for the Assign CRD.
- match: This allows us to filter the mutated resource by kinds, labelSelector, namespaceSelector, namespaces, and excluded namespaces.
Mutation Intent
This section contains the location to be mutated and the value of the mutation. Note that the value can be a composite value. This will be shown in an example later.
Conditionals: Conditionals will allow basing mutations on pre-existing values. As of gatekeeper version 3.4.0, this is still not implemented.
Installing Gatekeeper.
Gatekeeper mutation is in the alpha stage and as such, needs a few tweaks in the deployment.
Please use the following manifest to install gatekeeper with mutation enabled:
kubectl apply -f https://raw.githubusercontent.com/mmirecki/blog/main/gatekeeper/manifests/gatekeeper.yaml
A “gatekeeper-system” namespace should be created, containing the gatekeeper-controller-manager and gatekeeper-audit pods. A “gatekeeper-mutating-webhook-configuration” mutating webhook should also be present.
Installing gatekeeper using the gatekeeper operator
An alternative way to install gatekeeper is using the gatekeeper operator. The operator allows you to install and easily modify the gatekeeper configuration.
To install the operator, run:
kubectl apply -f https://raw.githubusercontent.com/gatekeeper/gatekeeper-operator/v0.1.2/deploy/gatekeeper-operator.yaml
The operator then needs a config to deploy gatekeeper. The config allows control of gatekeeper parameters such as gatekeeper pod node selectors, webhook failure policies, and more. An example of a config manifest with all values can be found here.
The simplest config enabling mutation can added by running:
kubectl apply -f https://raw.githubusercontent.com/mmirecki/blog/main/gatekeeper/manifests/gatekeeper-operator-config.yaml
Examples
Let's look at some use cases for using gatekeeper mutations.
Owner Annotation for All Pods
First, let’s add an owner annotation set to admin to each Pod created. Each created/updated pod will be mutated to have a field “metadata.annotations.owner" with the value “admin”. Below is the definition of an AssignMetadata for this mutation
apiVersion: mutations.gatekeeper.sh/v1alpha1
kind: AssignMetadata
metadata:
name: demo-annotation-owner
namespace: default
spec:
match:
scope: Namespaced
kinds:
- apiGroups: ["*"]
kinds: ["Pod"]
location: "metadata.annotations.owner"
parameters:
assign:
value: "admin"
To apply this mutation, run the following command:
kubectl apply -f https://raw.githubusercontent.com/mmirecki/blog/main/gatekeeper/manifests/examples/1_demo_annotation_owner.yaml
Create a pod by running:
kubectl create -f https://raw.githubusercontent.com/mmirecki/blog/main/gatekeeper/manifests/examples/pod.yaml
Examine the pod and note that an owner annotation has been added:
kubectl get pod demo -o yaml
Set securityContext.privileged=false for all pods
This example will add set securityContext.privileged to false for containers named “foo” in created pods.
Below is the definition of an Assign for this mutation. Note that the path specifies which containers the mutation will apply to: “spec.containers[name:foo]”
apiVersion: mutations.gatekeeper.sh/v1alpha1
kind: Assign
metadata:
name: demo-privileged
namespace: default
spec:
match:
scope: Namespaced
kinds:
- apiGroups: ["*"]
kinds: ["Pod"]
applyTo:
- groups: [""]
kinds: ["Pod"]
versions: ["v1"]
location: "spec.containers[name:foo].securityContext.privileged"
parameters:
assign:
value: false
To apply this mutations, run the following:
kubectl apply -f https://raw.githubusercontent.com/mmirecki/blog/main/gatekeeper/manifests/examples/2_demo_privileged.yaml
Create a pod by running:
kubectl create -f https://raw.githubusercontent.com/mmirecki/blog/main/gatekeeper/manifests/examples/pod.yaml
Examine the pod and note that the “foo” container now has securityContext.privileged set to false.
Adding a Sidecar
In this example, a sidecar container will be added to a created pod. Let’s assume we want to add a “networking” container as a sidecar to our pod.
Below is the definition of an Assign for this mutation. Note that the value in this case is a composite value, containing the definition of the container being added.
apiVersion: mutations.gatekeeper.sh/v1alpha1
kind: Assign
metadata:
name: demo-sidecar
namespace: default
spec:
match:
scope: Namespaced
kinds:
- apiGroups: ["*"]
kinds: ["Pod"]
applyTo:
- groups: [""]
kinds: ["Pod"]
versions: ["v1"]
location: "spec.containers[name:networking]"
parameters:
assign:
value:
name: "networking"
imagePullPolicy: Always
image: docker.io/centos
command: ["/bin/bash", "-c", "sleep INF"]
Examine the created pod, and note that a sidecar container was added.
