What do the planets of the solar system and Australian cybersecurity strategies have in common? There are eight of them! (Sorry, Pluto).
The Essential Eight is a set of baseline recommendations published by the Australian Cyber Security Centre (ACSC) that defines the minimum security controls that should be applied to protect systems. We have created OpenSCAP profiles for ACSC Essential Eight and Red Hat Enterprise Linux and released these with Red Hat Enterprise Linux 8.2. You can read more about the Essential Eight at the ACSC website.
This guide builds on our work with Red Hat Enterprise Linux and looks at the Essential Eight and Red Hat OpenShift Container Platform and how you can better protect OpenShift clusters and container applications using the Essential Eight strategies.
#1 Application Control
Application control is a critical component of the Essential Eight and protects against unauthorized application/process execution. Red Hat OpenShift supports controlling the registries that worker nodes can pull images from. This allows security and ops teams to create registries with validated and verified content that can be deployed to the platform.
Red Hat Quay 3.2 supports repository mirroring, which can be used to create an allow-listed source of container image content for Red Hat OpenShift users/operators. Repository mirroring synchronizes selected “upstream” container images into a local Quay instance. By ensuring that worker nodes can only pull content from this local Quay instance of verified, permitted content, you can significantly decrease the risk of vulnerable applications being deployed to the platform.
To support this content ingress model, you should also specify Quay as an allowed registry for Red Hat OpenShift to import images from. You can do this using the image controller resource and updating its parameters. Using the Red Hat OpenShift command-line interface edit the cluster image controller:
oc edit image.config.openshift.io/cluster
You can use both the allowedRegistriesForImport and allowedRegistries configuration options to specify which registries users can import images from. There is a subtle difference between these options:
- allowedRegistriesForImport limits the container image registries from which normal users may import images. It only affects the image API (so for example, oc import-image and ImageStreams).
- allowedRegistries determines how the container runtime (CRI-O) should treat individual registries when accessing images for builds and pods.
#2 Application and Operating System Patching
Platform application control
Red Hat OpenShift is built on Red Hat Enterprise Linux CoreOS (RHCOS), a container-optimized variant of Red Hat Enterprise Linux (RHEL).
RHCOS supports “controlled immutability.” Although it contains RHEL components, RHCOS is designed to be managed more tightly and indirectly than a default RHEL installation. Management is performed remotely from the Red Hat OpenShift Container Platform cluster, and on set up, RHCOS machines have only a few system settings that can be modified. This controlled immutability allows Red Hat OpenShift Container Platform to store the latest state of RHCOS systems in the cluster, so it is always able to create additional machines and perform updates based on the latest RHCOS configurations.
RHCOS’s support for controlled immutability means that software cannot be installed outside of the Red Hat OpenShift lifecycle, and only those limited processes required to run and manage container workloads are included in the OS.
SELinux is enforced by default on RHCOS worker and master nodes. SELinux does not control which binaries can execute, but instead determines which resources, like files and ports, a process can access. SELinux is like “social distancing for processes.” It keeps processes isolated from others, ensuring in the event of a compromise, the potential damage is significantly reduced.
Red Hat OpenShift makes further use of the security benefits of SELinux categories to create more secure containers. SELinux labels each container with a unique kernel-level context on the host, and these contexts isolate containers preventing direct container-to-container access. This means that even if a container is compromised and a container escape occurs, the attack is restricted to only that container; SELinux ensures that the compromised process cannot interact with other system services, like Unix domain sockets.
Patching Applications Deployed to Red Hat OpenShift
Operators are the reason that Red Hat OpenShift can support simple, over-the-air updates and can also be used to support application updates on the cluster.
Many Kubernetes operators support application updates for complex applications. For example, the Prometheus operator supports upgrading highly available Prometheus clusters on Red Hat OpenShift.
The Prometheus custom resource spec provides a field for the cluster version:;
You can upgrade the Prometheus cluster by simply editing the CR and changing the value of the ‘version’. For example, specifying version ‘v2.17.2’:
Once you update the cluster version in the Prometheus instance spec, you can see the operator start to pull down the cluster pods, upgrade to the latest version, and re-deploy.
#3 Configure Microsoft Office Macro Settings
This mitigation strategy is not applicable to Kubernetes/Linux environments. It is worth noting that Red Hat and Microsoft are bringing Windows services and containers to Red Hat OpenShift, allowing you to run applications like Microsoft SQL Server as a containerised service. You can also run Windows virtual machines on Red Hat OpenShift with OpenShift Virtualization, and benefit from all of the security features of a hardened Kubernetes platform.
#4 User Application Hardening
DevSecOps Processes for Application Hardening
DevSecOps applies the ideas of lean and agile to accelerate the creation of more secure systems and services. It extends beyond end-of-cycle checklists and “rubber-stamp security” to providing red and blue exploit testing, data and security science, and open contribution and collaboration to solving security challenges. DevSecOps processes allow developers and operators to better support application hardening on Red Hat OpenShift.
Tekton is an open source, Kubernetes-native approach to continuous integration and continuous delivery (CI/CD), and can be used to inject security into application build processes on Red Hat OpenShift such as injecting code signing, static code analysis, CVE scanning, and more advanced processes like vulnerability fuzzing. Tekton is managed using Kubernetes APIs and can be deployed to Red Hat OpenShift using the OpenShift Pipelines operator.
Once Tekton is available, you can use the Kubernetes APIs it exposes to inject tasks for specific security processes. For example, a Sysdig container image scan could be invoked using the following custom resource:
- name: image
- name: scan-image-step
- name: IMAGE_TO_SCAN
- name: SYSDIG_SECURE_TOKEN
In this example, we have abstracted the name of the image to scan, allowing this task to be reused across various build pipelines. You can then use this task in larger application build pipelines, such as the examples provided at the Tekton project.
#5 Restricting Administrative Privileges
Security Context Constraints, or SCCs, can greatly help to restrict the privileges available for container applications on OpenShift. SCCs control permissions for pods, including actions that a pod can perform and what resources it can access. You can use SCCs to define a set of conditions that a pod must run with to be deployed on OpenShift.
By default, Red Hat OpenShift runs pods using the ‘restricted’ SCC. The restricted SCC:
- Ensures that pods cannot run as privileged.
- Ensures that pods cannot mount host directory volumes.
- Requires that a pod run as a user in a pre-allocated range of UIDs.
- Requires that a pod run with a pre-allocated SELinux Multi-Category Security (MCS) label.
- Allows pods to use any FSGroup.
- Allows pods to use any supplemental group.
The restricted SCC provides sensible defaults, ensuring that pods cannot access privileged resources or data, and you should use it wherever possible. Some applications may not run in a restricted SCC and need additional privileges. It is possible to create a new SCC for the application, though you should carefully consider each additional privilege.
In particular, you should avoid the ‘RunAsAny’ SCC strategy, if possible. This will allow pods to specify the user ‘0’ and run as the root user, which should be avoided wherever possible. If you need to customize SCCs for a particular application, you can find more information in the Red Hat OpenShift docs.
#6 Multi-Factor Authentication
Red Hat OpenShift supports external OAuth and OpenID Connect integration with the web console and API, allowing users who use the Red Hat OpenShift command-line interface to also benefit from enhanced authentication/authorization. OAuth/OpenID connect integration also makes it much simpler to configure multi-factor authentication for the Red Hat OpenShift platform, as it means that this can simply be configured via the OAuth provider.
Red Hat OpenShift Container Platform 4 currently supports the following OAuth identity providers out of the box, all of which support multi-factor authentication:
- GitHub/GitHub Enterprise
Red Hat OpenShift also supports any OpenID Connect Provider that implements the OpenID Connect Discovery specification, such as auth0.
For on-premises Red Hat OpenShift deployments, which may require integration with ActiveDirectory or other LDAP databases, you can use Red Hat Single Sign On to federate the identity backend and present an OpenID Connect interface to Red Hat OpenShift.
#7 Daily Backups
The Essential Eight recommends daily backups to support recovery in the event of a cybersecurity incident that impacts application availability or access to data. Red Hat OpenShift supports several methods of backing up application and cluster data:
Operator-Enabled Database Backup
Just as Red Hat OpenShift components are managed by operators, they can also be used to manage complex persistent applications on a cluster. Operators are available to deploy and manage various databases on Red Hat OpenShift, such as MongoDB, Crunchy PostgreSQL and Apache CouchDB.
These operators often support automatically backing up the database cluster. For example, the Crunchy PostgreSQL operator includes support for full, incremental, and differential backups of highly available PostgreSQL clusters, as well as efficient delta restores. The operator also supports configuring how long database backups are retained for.
Etcd is a distributed key-value store and manages the state of a Red Hat OpenShift cluster. It is important that etcd is regularly backed up to ensure your cluster can be rapidly restored in the event of an incident.
To perform an etcd backup, start a debug session for a master node, change your root directory to the host, and run the cluster-backup script:
oc debug node/<node_name>
sh-4.2# chroot /host
sh-4.4# /usr/local/bin/cluster-backup.sh /home/core/assets/backup
In this example, two files are created in the /home/core/assets/backup/ directory on the master host:
- snapshot_<datetimestamp>.db: This file is the etcd snapshot.
- static_kuberesources_<datetimestamp>.tar.gz: This file contains the resources for the static Pods. If etcd encryption is enabled, it also contains the encryption keys for the etcd snapshot.
If etcd encryption is enabled, it is recommended to store this second file separately from the etcd snapshot for security reasons. However, this file is required in order to restore from the etcd snapshot.
#8 Platform Hardening
Red Hat OpenShift is inherently designed to be deployed across multiple public cloud providers, creating an open hybrid cloud that supports greater application portability and lower costs. Securing these clusters and ensuring their configuration remains consistent can be challenging, and Red Hat is collaborating with the “Compliance as Code” open source project to address this.
We’re creating “Compliance as Code” evaluation checks that can verify the cluster state against the Essential Eight requirements. For example, checking whether allowedRegistries are configured, or whether there are authentication providers configured that lack support for multi-factor authentication. This content is designed to be used with the Compliance operator, a Kubernetes operator that can scan Red Hat OpenShift cluster configuration and apply remediation.
We’re actively developing this content, and you can provide feedback and contributions directly to us via the open source “Compliance as Code” project. Developing this content in the open means that everyone can contribute to creating a more secure platform, aligned with the Essential Eight.
OpenShift Container Platform, How-tos, OpenShift 4