Making the local development experience with OpenShift as easy as possible is one of my main concerns. I’m constantly exploring how to enhance our current experience as I develop applications for OpenShift. I work hard to understand developers’ requirements and eventually provide solutions in the tooling we provide. I incubate ideas in a project my team owns, oc-cluster-wrapper. Depending on the nature of the problem, I work very closely with our engineering teams to solve these use cases in “oc cluster” or “Minishift,” as they have different goals, even if they both can stand up an OpenShift all-in-one instance for local development.
Minishift/CDK is the tool that has been created for the end user (that is, developer) of OpenShift, to streamline and simplify many of the common tasks a developer will perform on the platform. This extensible tool has recently introduced the notion of “addons” to allow users and organizations to provide common bootstrapping to their clusters. (I’ll cover addons in another post.)
In this post I want to focus on something that really annoys me, as a developer, and I will share the solution I have found for this problem. As always, if the community sees that this is convenient for the general user (the developer), we will make sure that Minishift implements this solution out of the box.
By now you may be wondering what my problem is. Let me describe it.
The problem
I create and destroy oc clusters on a daily basis. When I want to work on a branch, on a feature, or on a demo, I create a new oc cluster, and when I have finished working on that task, and I know I will no longer work on it, I destroy it. Every time I do this, and I access the OpenShift web UI, I am prompted to accept a certificate. OpenShift’s web UI is exposed through https, and oc cluster uses self-signed certificates for this communication.
Once the certificate has been accepted, I will not be prompted again. Although the communication is not trusted, I have voluntarily agreed to trust that certificate.
If I open a different browser, I will be prompted again to “proceed” understanding the risks that using a self-signed certificate implies.
Now, I’m not truly accepting any random certificate on the interwebs managed by who knows who. I’m a developer who has created a cluster on my local machine for self-use. That means what I’m really doing is trusting me, which is something that I usually do.
The second problem comes when I look into how oc cluster works, and I see that every time I stand up a new all-in-one instance via oc cluster up I get a new certificate. Even though I may have already accepted a certificate before for a previous instance, I will again be prompted to trust another certificate.
I’m a mostly Java developer and I’m not an expert on certificates, so after digging a little bit on the internet I learned that a certificate is signed by what is called a “Certificate Authority,” which is an entity that is “globally trusted” and validates that who is using the certificate can also be trusted. This is known as a “chain of trust.”
The process of having a CA validating who you are and what you do so you can be trusted is a complex and costly process; most of the time that’s not convenient for ephemeral certificates, those that will live for some time, and are somehow meant for development or testing purposes. That's why oc cluster provides its own Certificate Authority (CA) to use to create all the certificates it will require when creating a cluster. This Certificate Authority is also created with every new oc cluster.
What can I do then?
The first and easiest option is to create a CA myself and provide it to oc cluster so that every certificate that gets created is signed by this Certificate Authority. I can then add this CA to the CAs I trust in my laptop, so I will never be prompted again to accept a certificate that is signed by it. This is fairly easy to do as OpenShift provides a convenient command that can be used to create a CA:
$ oc adm ca create-signer-cert \
--cert "my-ca.crt" \
--key "my-ca.key" \
--serial "my-ca.serial.txt" \
--name="jorge@localhost"
Then you can just provide this CA to the oc cluster up command line, and it will be used.
$ oc cluster up --certificate-authority=my-ca.crt
This is a really easy solution, but it falls short. I got educated by one of my colleagues about the risks of this option: If I globally trust this CA in my laptop and somehow this CA gets leaked from my laptop to someone else, he could just use it for malicious purposes that I would blindly “trust.” This is not a good idea.
My colleague explained to me that a better idea would be to create a certificate to only reuse between all my oc clusters, and that this certificate is what I should globally trust on my laptop. This wouldn’t generate any security risk at all.
OK, let’s explore this option.
One of the characteristics of oc clusteris that in the bootstrapping process, if there’s already a configuration for the instance, it will reuse it (unless it’s not compatible). So, the only thing I need to do is, instead of creating a CA, I need to create the certificates that the web UI will use whenever I access it.
It happens that there are multiple certificates used by an OpenShift instance, and that the CA that will sign these certificates will be different from instance to instance unless reused. So eventually, I need to create the CA and all the certificates, and provide these to every instance I create.
For this purpose, there is again an oc command:
$ oc adm ca create-master-certs \
--cert-dir=./certs \
--master=https://127.0.0.1:8443 \
--public-master=https://127.0.0.1:8443 \
--hostnames=kubernetes,kubernetes.default,kubernetes.default.svc,kubernetes.default.svc.cluster,kubernetes.default.svc.cluster.local,localhost,openshift,openshift.default,openshift.default.svc,openshift.default.svc.cluster,openshift.default.svc.cluster.local,127.0.0.1,172.17.0.1,172.30.0.1,192.168.65.2
As you can see, this command needs some quite different information. I will indicate the directory where to leave all the certificates, then the API server’s internal and external URLs, and also a list of hostnames or IPs that the server certificates should be valid for. This last part is the only challenging one, as it requires me to reuse the same IPs and names to be used for the cluster as part of the --public-hostname argument at bootstrapping time.
Now, every time I bootstrap a cluster, all I need to do is provide it with all these certificates. In order to do that, I place all these certificates in a well-known location on my laptop and then instruct oc clusters bootstrapping process to preserve the configuration and to use this “well-known” location for the master’s config files.
The following options will be required whenever I start an oc cluster:
$ oc cluster . . . --host-config-dir=<DIR>/config --use-existing-config
Every cluster I bootstrap from this point will use the same certificates, so the only part left is to instruct my local development system (my laptop) to trust this certificate for https communication.
As I use macOS, I will register my certificate using KeyChain. You can directly double-click on the certificate you want to add and then trust the certificate. In this case it will be master.server.crt. You can see the process in the following animated image. (There’s a similar process for every operating system.)
From this point forward you will never be asked about this certificate by your browser; and you can create as many clusters as you want, as long as you share this certificate to all of them.
Making it simple
As you can see, making this tip work with oc cluster is not trivial. For that, my team has been working on a script that makes working with oc cluster locally really simple. The script is called “oc-cluster” and works in the same way as the base command oc cluster, but in a simpler way.
All the boilerplate that I have presented here is already built into that script, and you can just create a cluster by:
$ oc-cluster up
The only step left to you is to trust the certificate. If you do, the burden of being asked for every cluster you create will be removed.
All the convenience this script provides (which is not the topic of this blog), is used as incubation for developer usability requirements that will get introduced into Minishift, meaning that eventually Minishift will support similar functions.
Wrap up
Here I have presented a way of making your local development experience easier by allowing you to avoid the nagging behaviour of having to accept the self-signed certificate every time you create a cluster with oc cluster.
You can make a lot of small improvements to improve your day-to-day experience. That’s what we’re exploring and contributing into Minishift. Minishift is the most streamlined way of working locally with OpenShift clusters for developers and what I recommend using. And although some of the ideas we experimented with in oc-clusterare not yet implemented, they will be soon.
But remember, Minishift is not a “production ready cluster” or a tool for “operations.” It is just meant to be used as the cluster where developers can test their applications early in the process. This is very important to remember. The experience in Minishift is already and will be further streamlined for development.
I would like to hear from you! If you have some ideas or comments, please do not hesitate to tweet me your comments.
Categories
