Foreword
This post is meant to be a braindump - to present a common understanding of the different software development/consumption models our customers have.
On different types of Containers
In general, we distinguish three different types of containerized application components:
- System and Standalone Containers - used to deliver operating system related or hardware related services. In some cases they may be started even before the docker daemon is running.
- Stateless, Isolated, batch-like Containers - executing very uniform workloads, may or may not be orchestrated by Kubernetes and run most often on a cluster scale.
- Stateful Containers - most often orchestrated by Kubernetes and tightly bound to a configuration state and data storage
A more detailed elaboration on the Workload Characteristics and Candidates for Containerization is available on the Red Hat Enterprise Linux blog.
A few examples…
A so-called Service Container is a container that provides services (like a database) to other components of an application. This database service container is not only using persistent storage, to save a state between restarts of the container, it also requires a tightly bound specific storage, as each of the database nodes (implemented in a different container) requires a specific storage (eg directory with node specific information). This could be classified as a stateful container.
Another example of a stateful container is a so-called Pet Container: a container that has been configured by a system administrator, software or configuration has been changed during the life cycle of the container itself, and not only during the build time of the container image.
Using a System Container a host can be assembled from core system services running in a container. These can be DNS, DHCP, syslog, udev, etc. The main pain points here are:
- It may not be trivial to run such service in a container
- An operator needs to ensure that the service (=container) is started at the right time (you want logs as soon as possible, right?) and order (etcd before kubernetes master, right?)
- Some of these containers are likely needed to be up before container runtime (and container orchestrator) are started
- These containers require a lot of privileges so it may be hard to have the security right
- Update model, configuration, management — these are not clear, yet
Stateless or stateful containers may deliver some service as part of a multi-container application, for example, an ephemeral caching key-value store or a persistent database. Or they provide application logic written by the user and compiled into a container image running on top of the container platform.
Introduction
We can assume, that a company will not only use software written by itself but also use ISV software and put additions on top of software from other sources (may it be additional functionality or integration layers).
The following text will elaborate on the models software may be developed and brought to production. At present, we can see that no ‘pure mode’ will be used by a company, it will always be mixes and blends of modes, modes can even vary from component to component.
Four Modes how Software and OpenShift interact:
- Created Source Code, by the company, built and run on OpenShift (S2I with git source or a docker build from a Dockerfile)
- Binary Artifacts built outside of OpenShift, OpenShift packages them into containers (S2I with binary artifact) and runs them
- Container images built outside of OpenShift and run by OpenShift
- Company provided software, built on top of OpenShift but run elsewhere
These modes are applicable per component of the larger software system. Example: database software is consumed from an ISV, that ISV delivers a set of container images, ACME Corp puts some integration code on top of these images and deploys it to production on top of OpenShift, we call this Mode B. Another component of the same system is built from source and run in production entirely on top of OpenShift, this is Mode A.
The same patterns can be applied to the components used for managing a software lifecycle:
- OpenShift hosts the CI/CD services, either as a central service or per project
- CI/CD is hosted outside of OpenShift and orchestrates the workflow inside OpenShift
Organizations that have a well-established CI/CD infrastructure may not want to move these services onto of OpenShift, but they require the same level of tight integration between CI/CD and OpenShift as a solution that is completely hosted on top of OpenShift.
General Questions
What is the general workflow?
The following chapters will describe a workflow per Mode. As mode A and D are quite similar we will join them into one workflow description. Each description is accompanied by an abstract prototypical architecture to implement the workflow described.
Mode A/D - Software built and run on OpenShift or external to OpenShift
In these two modes, a software stack is composed either from source code created by the company and/or based on source code and libraries from external contributors. The application - as a set of container images - is compiled on OpenShift, services of the platform (BuildConfigs with S2I, Triggers, ImageStreams) are used to optimize the build process, the resulting set of container images is stored and either run on OpenShift or distributed to other container runtime environments. These modes are applicable for both corporate software development and for ISVs.
Mode B - Artifacts built outside but run on OpenShift
An existing build/CI/CD infrastructure is leveraged to work with OpenShift, artifacts are build and life cycled by this CI/CD service and deployed to OpenShift. The result of CI/CD will be container images that are stored in an enterprise registry. These artifacts are then pulled into the OpenShift environment and are deployed for production. In this mode, only the target platform is OpenShift, never the less, build artifacts must be packaged into container images to be deployed on OpenShift. This final transformation step utilized Source-to-Image, BuildConfigs, and ImageStream to create the final container images.
Mode C - Container images built outside but run on OpenShift
This mode assumes that a build/CI/CD infrastructure is in place and used to generate container images. In contrast to Mode B, this infrastructure creates the final container images to be deployed to OpenShift. The resulting container images must be stored in an enterprise registry and will be deployed to OpenShift from there. There is no specific requirement if a CI/CD infrastructure life cycled these images or not. Even a developer may build the images on his laptop and push them to the enterprise registry.
Side Note: We as Red Hat focus on OpenShift as a target platform. At least Mode C workflows can target other platforms.
What features of OpenShift are used per each specific mode?
Decision criteria:
- Is it available pre-packaged?
- Can we create and maintain this component on our own?
How can I transition from one mode to another?
Transitioning from one mode to another may happen on a per component basis: If a CI/CD infrastructure is in place, there is no need to migrate it on top of OpenShift, but it could leverage services provided by OpenShift. The goal should be: provide as many of the CI/CD on top of OpenShift, as a Jenkins Pipeline based CI/CD offers the most flexible solution, especially with regards to resource utilization, as this solution will profit from OpenShift workload scheduling of Pods.
To be more specific, let's have a look at an example:
Your company has a team of operators providing a Jenkins cluster on VMs, developers can use theses Jenkins cluster to run build jobs. Jenkins operations are controlling which plugins are installed and how they are configured. Developers may use a Jenkins Pipeline or Jenkins Jobs.
With this example, the central theme is: DevOps will provide CI/CD as a service, may it be running on VM or be moved on top of OpenShift (as a central or per project Jenkins instance).
So what does a transition from an old infrastructure point of view on Jenkins to a new DevOps-like CI/CD as-a-service look like?
The Ops team will containerize Jenkins first, using a Source-to-Image builder image of Jenkins to create Jenkins container images containing all the plugins and configurations the company uses or requires to operate Jenkins. These container images could be used by any Dev within the company to run his build pipeline on the standard Jenkins software stack that is used by Ops. In fact, Ops is controlling and providing the standard Jenkins software stack, and no Developer needs to care about it anymore. No surprises will happen to the developer as he moves his tests from his development environment to the test environment or later to staging.
Categories
