Introduction

Let's Encrypt is a global Certificate Authority (CA) provider. It offers domain validation certificates and allows organizations to obtain, renew, and manage SSL/TLS certificates. You can request a certificate and use it as a valid SSL/TLS certificate on your website or application requiring a valid SSL/TLS certificate, without the need to generate a self signed certificate. All this is automated and provided free of charge.

In short, you can request and use SSL/TLS certificates for free!

One important caveat is that the certificates are only valid for 90 days, but the renewing of certificates is easily done and can be automated.

In this post, we will be exploring how to use these certificates when you're testing out OpenShift in your environment. I'm going to assume the following (that is, these are the prerequisites):

  • You have OpenShift (either Enterprise or Origin) running.
  • You installation is available on the internet. That is, it's "public" (more on this later).
  • You have a router already deployed on your OpenShift installation.

Setup

Certbot is a command line utility that automates the creation and renewal of SSL/TLS certificates on Let's Encrypt. This utility is available on the EPEL repositories; so it can be installed on any Fedora/EL7 system. The installation is straightforward. You need to run this installation on the node (or one of the nodes in a High Availability (HA) installation) that is running your router. Below is an example of my installation.

# oc get pods
NAME READY STATUS RESTARTS AGE
docker-registry-3-mjkqz 1/1 Running 0 2d
registry-console-1-0q9xs 1/1 Running 0 2d
router-3-rw077 1/1 Running 0 2d

# oc get pod router-3-rw077 -o jsonpath='{.spec.nodeName}{"\n"}'
ip-172-31-25-173.us-west-1.compute.internal

As you can see my router is running on node ip-172-31-25-173.us-west-1.compute.internal, so I'll install certbot there. The installation is not too complicated.

If you're running OpenShift on Fedora:

dnf -y install \
https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm
dnf -y install certbot

If you're running OpenShift on EL7 (RHEL/CentOS):

yum -y install \
https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm
yum -y install certbot

Use Case 1: Greenfield Applications

I'll start with the most simple and straightforward use case: A greenfield application. There are 4 basic steps:

  1. Scale your router to 0.
  2. Request the certificate.
  3. Scale your router back to 1.
  4. Upload your certificate.

Before we get into it, let's create a test project and app to test with:

# oc new-project test
# oc new-app openshift/php~https://github.com/RedHatWorkshops/welcome-php -n test

Requesting Your Certificate

Certbot creates a "dummy" web server when requesting a certificate; it does this in order to verify the certificate after it's been created. Because this is done on port 80, it will interfere with the router. As we are only testing this on a dev system, we'll scale the router down to avoid a port conflict:

# oc scale dc/router --replicas=0 -n default
deploymentconfig "router" scaled

Now let's request the certificate; the details below should be self-explanatory. Replace with your domain and email where appropriate:

certbot certonly \
--standalone \
-d welcome-php.apps.example.com \
--agree-tos -m example@example.com

If you get stuck, use certbot --help as it's well documented.

This will give you a "congrats" message when successful. Once you see this, scale your router back to 1 (or whatever number it was before):

# oc scale dc/router --replicas=1 -n default
deploymentconfig "router" scaled

# oc get pods
NAME READY STATUS RESTARTS AGE
docker-registry-3-mjkqz 1/1 Running 0 2d
registry-console-1-0q9xs 1/1 Running 0 2d
router-3-qrq9z 1/1 Running 0 1s

The certificates (along with a README) is under /etc/letsencrypt/live/welcome-php.apps.example.com/:

# ls -1 /etc/letsencrypt/live/welcome-php.apps.example.com/
cert.pem
chain.pem
fullchain.pem
privkey.pem
README

Uploading Your Certificate

Now, create a route offloading SSL into the router. First, find out what service you want to point to:

# oc get svc -n test
NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE
welcome-php 172.30.157.92 8080/TCP 12m

Next, create the route uploading the certificate. Below I used "Redirect" for non HTTPS traffic:

oc create route edge welcome-php \
--service=welcome-php \
--cert=/etc/letsencrypt/live/welcome-php.apps.example.com/fullchain.pem \
--key=/etc/letsencrypt/live/welcome-php.apps.example.com/privkey.pem \
--hostname=welcome-php.apps.example.com \
--insecure-policy=Redirect -n test

The fastest way to test this is with curl:

# curl -I https://welcome-php.apps.example.com
HTTP/1.1 200 OK

Notice how you didn't have to use -k for insecure? Go ahead and visit the browser and see that the SSL is valid!

Renewing

These SSL certificates are only good for 90 days and it's recommended you renew every 60. You can set up a cron every 30 days to renew your cert. Running it before its renewal date is okay as it doesn't do anything if it's not due yet.

# certbot renew
Saving debug log to /var/log/letsencrypt/letsencrypt.log

-------------------------------------------------------------------------------
Processing /etc/letsencrypt/renewal/welcome-php.example.com.conf
-------------------------------------------------------------------------------
Cert not yet due for renewal

The following certs are not due for renewal yet:
/etc/letsencrypt/live/welcome-php.example.com/fullchain.pem (skipped)

Your cron script should perform the following:

  • Scale the router to 0.
  • Run the renewal.
  • Scale the router to 1.
  • Delete the old route.
  • Re-create the route; uploading the certificates.

In short:

# oc scale dc/router --replicas=1 -n default
# certbot renew
# oc scale dc/router --replicas=1 -n default
# oc delete route welcome-php -n test
# oc create route edge welcome-php \
--service=welcome-php \
--cert=/etc/letsencrypt/live/welcome-php.apps.example.com/fullchain.pem \
--key=/etc/letsencrypt/live/welcome-php.apps.example.com/privkey.pem \
--hostname=welcome-php.apps.example.com \
--insecure-policy=Redirect -n test

Use Case 2: Brownfield Applications

"But what if I have an already running application with a self-signed certificate?" you may ask. You can easily delete the route and add the generated certificate.

But let's take a more complex use case.

When you deploy OpenShift you can have the installer deploy metrics for you on install. This provides the following:

  • Metrics stack running in OpenShift.
  • Self-signed SSL certificate for the stack.
  • Offloads the Hawkular API's SSL cert with the router's default certificate.

What we want to do is replace the default router cert (which is self-signed) with a new certificate. The backend certificate can stay in place.

Requesting Your Certificate

The process to request the certificate is the same.

First, scale your router to 0:

# oc scale dc/router --replicas=0 -n default
deploymentconfig "router" scaled

Take a look at the routes to see what URL you'll be requesting:

# oc get routes -n openshift-infra -o jsonpath='{.items[*].spec.host}{"\n"}'
hawkular-metrics.apps.example.com

Request the cert with this hostname/url:

# certbot certonly \
--standalone \
-d hawkular-metrics.apps.example.com \
--agree-tos -m example@example.com

Once you get the "success" message; scale the router back to 1:

# oc scale dc/router --replicas=1 -n default
deploymentconfig "router" scaled

Verify the certificates are on the server:

# ls -1 /etc/letsencrypt/live/hawkular-metrics.apps.example.com/
cert.pem
chain.pem
fullchain.pem
privkey.pem
README

Uploading Your Certificate

If you take a look at the tls termination, you'll see that it's set to reencrypt:

# oc get routes -n openshift-infra \
-o jsonpath='{.items[*].spec.tls.termination}{"\n"}'
reencrypt

This means that the HA Proxy router takes the request and reencrypts it before it sends it back to the backend service address, using its own CA certificate. You can export this CA certificate. Take a look at this by running:

# oc get secrets hawkular-metrics-certificate -n openshift-infra \
-o jsonpath='{.data.hawkular-metrics-ca\.certificate}' | base64 -d

You can redirect this to a file to use it later:

oc get secrets hawkular-metrics-certificate -n openshift-infra \
-o jsonpath='{.data.hawkular-metrics-ca\.certificate}' | base64 -d > ~/hawkular-metrics.ca

Now that you have this certificate exported, delete the current route:

oc delete route hawkular-metrics-reencrypt -n openshift-infra

Use the exported CA cert to create a new route with the new certificates:

oc create route reencrypt hawkular-metrics-reencrypt -n openshift-infra \
--hostname hawkular-metrics.apps.example.com \
--cert /etc/letsencrypt/live/hawkular-metrics.apps.example.com/fullchain.pem \
--key /etc/letsencrypt/live/hawkular-metrics.apps.example.com/privkey.pem
--service hawkular-metrics \
--insecure-policy=Redirect \
--dest-ca-cert ~/hawkular-metrics.ca

Just like before, you can cron this to renew the certificate every other week or so.

Conclusion

In this post, we showed you how to use Let’s Encrypt to deploy and test valid SSL certificates in your environment (you can also test them with the SSL Server Test). Because these are temporary certificates, we don’t recommend doing this for a production installation (you wouldn’t want to scale your router down anyway).

This shows how OpenShift has an easy way to manage SSL certificates with the oc command line utility. Also, OpenShift provides many methods for SSl termination including reencrypt and edge.

You can use Let's Encrypt to test different scenarios on a test system before you go and create permanent SSL certificates.


About the author

Christian Hernandez currently leads the Developer Experience team at Codefresh. He has experience in enterprise architecture, DevOps, tech support, advocacy, software engineering, and management. He's passionate about open source and cloud-native architecture. He is an OpenGitOps Maintainer and an Argo Project Marketing SIG member. His current focus has been on Kubernetes, DevOps, and GitOps practices.

Read full bio