In OpenShift 3.1 we introduced the concept of jobs. A job will create one or more pods and run them to completion. This is different from a replicationController, which keeps the pods up and running. A job will run the pod until the pod finishes the task it was assigned. You can think of a job as a pod with a restartPolicy of Never.

A job will track the progress of the assigned task and then will update with the status: active, succeeded, or failed.

A few use cases for using a job:

  • Batch Process That need to run at regular intervals
  • Distributed processing of very large data sets
  • Long running process for system maintanace

Job Object

The configuration of a job is done by using the "Job" object type. An example would look like this:

apiVersion: extensions/v1beta1
kind: Job
metadata:
name: pi
spec:
selector:
matchLabels:
app: pi
parallelism: 1
completions: 1
template:
metadata:
name: pi
labels:
app: pi
spec:
containers:
- name: pi
image: perl
command: ["perl", "-Mbignum=bpi", "-wle", "print bpi(2000)"]
restartPolicy: Never

A few key points here...

  • The selector references the label selector and it sets up where the job will run
  • parallelism is an optional parameter for how many pods to run at the same time for the task (defaults to completions)
  • completions states how many times the pod needs to run successfully in order to be marked succeeded (defaults to one if not specified)
  • The created template for the pod the controller.

Creating a Job

We can use this example (which just counts Pi to 2,000 digits) to see how to create a job. I have saved the example as pijob.yaml and use oc create to create the job:

$ oc create -f pijob.yaml
job "pi" created

List job status with the oc command...

$ oc get jobs
JOB CONTAINER(S) IMAGE(S) SELECTOR SUCCESSFUL
pi pi perl app in (pi) 0

If list your pods you should see it in the "pending" state

$ oc get pods
NAME READY STATUS RESTARTS AGE
pi-169h2 0/1 Pending 0 31s
welcome-php-1-be674 1/1 Running 1 7d
welcome-php-2-build 0/1 Completed 0 7d

After the image gets pulled into the node; the status will change to "running"

$ oc get pods
NAME READY STATUS RESTARTS AGE
pi-169h2 1/1 Running 0 7m
welcome-php-1-be674 1/1 Running 1 7d
welcome-php-2-build 0/1 Completed 0 7d

If you are logged into the Web UI; you should see the job appear in the overview page
jobs in OpenShift

Once the pod is completed; you can see the status of the job

$ oc get jobs
JOB CONTAINER(S) IMAGE(S) SELECTOR SUCCESSFUL
pi pi perl app in (pi) 1

View the logs to see the output of the job

$ oc logs pi-75ssl
3.1415926535897932384626433832795028841971693993751058209749445923078164062862089986280348253421170679821480865132823066470938446095505822317253594081284811174502841027019385211055596446229489549303819644288109756659334461284756482337867831652712019091456485669234603486104543266482133936072602491412737245870066063155881748815209209628292540917153643678925903600113305305488204665213841469519415116094330572703657595919530921861173819326117931051185480744623799627495673518857527248912279381830119491298336733624406566430860213949463952247371907021798609437027705392171762931767523846748184676694051320005681271452635608277857713427577896091736371787214684409012249534301465495853710507922796892589235420199561121290219608640344181598136297747713099605187072113499999983729780499510597317328160963185950244594553469083026425223082533446850352619311881710100031378387528865875332083814206171776691473035982534904287554687311595628638823537875937519577818577805321712268066130019278766111959092164201989380952572010654858632788659361533818279682303019520353018529689957736225994138912497217752834791315155748572424541506959508295331168617278558890750983817546374649393192550604009277016711390098488240128583616035637076601047101819429555961989467678374494482553797747268471040475346462080466842590694912933136770289891521047521620569660240580381501935112533824300355876402474964732639141992726042699227967823547816360093417216412199245863150302861829745557067498385054945885869269956909272107975093029553211653449872027559602364806654991198818347977535663698074265425278625518184175746728909777727938000816470600161452491921732172147723501414419735685481613611573525521334757418494684385233239073941433345477624168625189835694855620992192221842725502542568876717904946016534668049886272327917860857843838279679766814541009538837863609506800642251252051173929848960841284886269456042419652850222106611863067442786220391949450471237137869609563643719172874677646575739624138908658326459958133904780275901

Scaling

You can scale the job after it's creation by using the oc scale command. This modifies the parallelism dymanicaly while your job is running according to what you pass to the --replicas option.

$ oc scale job pi --replicas=3
job "pi" scaled

Summary

In this article we explored the job object, talked about some use cases, and ran through an example in which we also scaled the job.

 

Author

Christian Hernandez
Solution Architect
US CSO Solution Architect- OpenShift Tiger Team

@christianh814


About the author

Christian Hernandez currently leads the Developer Experience team at Codefresh. He has experience in enterprise architecture, DevOps, tech support, advocacy, software engineering, and management. He's passionate about open source and cloud-native architecture. He is an OpenGitOps Maintainer and an Argo Project Marketing SIG member. His current focus has been on Kubernetes, DevOps, and GitOps practices.

Read full bio